Smart Glasses Are Going to Work This Time, Google's Android President Tells CNET

In a secluded room inside Google’s New York offices, CNET’s Scott Stein browsed a bookshelf filled with titles he’s never read. After picking up one of the books, Absolution by Jeff VanderMeer, he asked whether he needed to read the other installments in the series to appreciate it. A friendly voice, coming from his glasses, told him that even though the book is set in the same universe as the author’s other works, it’s a standalone story.

Several minutes and many interactions later, when he asked about the books he previously browsed, the assistant remembered and recited them all. 

In a rare one-on-one conversation with CNET, Sameer Samat, Google’s president of the Android ecosystem, made it clear that he thinks exchanges like these will increasingly become the norm thanks to the arrival of Android XR, a collaborative effort between Google, Samsung and Qualcomm, unveiled on Thursday. The three tech behemoths, which together form the backbone of the Android smartphone landscape, are hoping to push Android into its next era with a new software platform designed to power mixed reality headsets, smart glasses and everything in between. 

Samsung is developing the first device — a headset codenamed Project Moohan — which will be available next year as a precursor to smart glasses that the company is also working on. It comes after Google announced on Wednesday that it’s releasing prototype smart glasses to testers to gather feedback. Google and Samsung also hosted developer demos in New York this week in hopes of convincing app creators to build custom experiences for the new software.

The smartphone has defined the last 15 years of our lives, and tech giants have failed to create a device as impactful since. Google, among others, has certainly tried. First there was Google Glass, the augmented-reality eyewear that ultimately flopped because of its high price, privacy concerns and technical limitations. Then came Google Daydream, a virtual reality platform that turned smartphones into VR machines, which the company discontinued in 2019 after slow adoption. 

The AR/VR market is still struggling five years later, with virtual and augmented reality shipments dropping 28.1% year-over-year in the second quarter of 2024, according to The International Data Corporation. Apple also reportedly cut its Vision Pro orders, according to The Information and TF International Securities analyst Ming-Chi Kuo, which some have interpreted as a signal of weak demand. It’s perhaps a sign that even a company credited with inventing the modern smartphone may be struggling to make mixed reality succeed. 

But Samat is certain that things will be different this time. The virtual assistant we’ve been waiting for, one that’s worth donning a pair of glasses or a visor for because it’s that helpful, is finally here, he says. And of course, it’s all because of advancements in artificial intelligence, particularly generative AI, which has upended the technology industry since its arrival in OpenAI’s ChatGPT in 2022.

“We have had attempts here before, but one thing that was clear was that the technology wasn’t quite ready,” Samat said. “The other thing, though, is we never stopped believing in the vision around all this, and we actually never stopped working on it either.” 

Read more: Why Apple’s Future Could Depend on Siri

Google and Samsung think AI can finally make mixed reality a hit

After trying Google’s prototype glasses and Project Moohan, which looks more like a traditional VR headset, it’s easy to understand why AI has been the missing piece of the puzzle. While they aren’t the first head-worn gadgets with virtual assistants, the experience is vastly different from what you’ll get from using Siri on Apple’s Vision Pro or the AI assistant in the Meta Quest. And, while Meta’s Ray-Bans also have camera-assisted AI functions already, they’re not as continuously aware yet as Google’s demos were.

The version of Gemini in Android XR isn’t like the question-and-answer bots of years past. It’s not just playing music on command, telling you the weather forecast, launching apps, reciting notifications or managing the temperature on your smart home thermostat. It’s actually seeing the world around you and making observations, almost like an actual human. Google teased this approach back in May when it unveiled Project Astra, an earlier prototype of the glasses-based assistant.

During the Project Moohan demo, CNET’s Scott Stein explored a virtual version of the Spotify Camp Nou soccer arena in Barcelona through Google Maps. All it took was a simple statement like, “Show me the most famous goals made here,” to pull up YouTube clips from soccer games played in that very location. When wearing the prototype glasses using Project Astra, Gemini was able to provide CNET’s Lisa Eadicicco with cocktail ideas based on the bottles she’d been looking at on the shelf in front of her.

This iteration of Gemini is more present and aware than other implementations that have existed so far. Once you activate Gemini on the glasses, it passively listens for requests, making it feel more like you’re having an ongoing conversation rather than using the assistant for siloed commands. Tapping the glasses’ arm pauses Gemini.

“We were playing around with what these models can do using the phone, and the cameras on the phone, as a way of interacting with the world,” Samat said. “And it was truly blowing us away, what was possible.”

There’s a chance privacy woes will arise, just like they did with Google Glass, and for good reason. Concerns over the way tech companies handle personal data and the outsized influence these companies have in our lives has reached an inflection point in recent years as regulators crack down on tech giants. New devices like smart glasses and immersive headsets may only serve to deepen those worries in the future.

In a broader press briefing, Samat said Google was working through “special privacy controls” for head-worn devices that the company will share more details about next year. 

“We fully understand and agree that [privacy] needs to be carefully addressed,” Samat said to a group of journalists. “And in addition, I would say it’s not just for the individual that has the glasses or the device on, but for the folks around as well.”

Ever since ChatGPT arrived in late 2022, there’s been an AI gold rush to infuse more products with chatbots, conversational interfaces and generative capabilities. Smartphones are among the biggest examples; companies like Google, Apple and Samsung have introduced more natural-sounding virtual assistants along with new tools for creating images based on prompts and summarizing text. Google also brought a feature called Circle to Search to Android in 2024, which lets you launch a Google search just by circling something on your device’s screen. It’s bringing a real-world version of that to Android XR, too. 

But these software features haven’t meaningfully resonated with consumers yet, nor have they been enough to move the needle in terms of smartphone sales. AI-first gadgets like the Humane AI Pin and Rabbit R1 handheld device, both of which are meant to primarily be interacted with via voice, also fell flat this year after failing to live up to expectations. 

Glasses and headsets have the distinct advantage of being placed right within your line of sight, which could make them more intuitive and appealing than smartphone-based efforts, Google and Samsung believe. Even though Google, Samsung and Qualcomm announced their partnership back in February 2023, progress in AI caused them to reevaluate their approach in the middle of development, according to Samat.

“With glasses, it really is the opportunity to engage with an AI assistant in the real world,” he said. “We think that application can be what email and texting was for the phone, for glasses. We think that that’s sort of a killer app in many ways.”

That vision will start with headsets, not glasses, given that these larger, visor-style devices can provide a foundation that glasses can eventually use.

“There’s perception technology, there’s world locking, there are a number of things that we feel like made sense to focus on with the headset and get right,” Samat said.

If you look at the history of Google’s product launches over the past decade, you can see the fragments of Android XR coming together. The glanceable widgets on Wear OS smartwatches (a platform it also created in collaboration with Samsung), Google’s history of bringing augmented-reality based features to smartphone apps and the voice-first interactions on smart speakers like the Nest Audio — it feels like there are pieces of them all in Android XR. Gemini is the thread tying them together. 

Wrist-worn wearables have an especially important role to play in Android’s expansion into mixed reality. While smartwatches are inherently different from smart glasses, they’re both optimized for showing bite-sized bits of information in a quick way. They also require computing to be split between the device itself and your phone. That’s why the Wear OS and Android XR teams closely collaborate, Samat says.

Smartwatches and smart glasses seem poised to work together. It’s easy to imagine a future in which your smartwatch enables new types of gesture inputs, or perhaps works in conjunction with your glasses to display health and fitness statistics, the latter of which Samat hinted could be a potential use case in the future.  

None of those scenarios are a reality yet, and Samat couldn’t say when or if they will be. But that’s part of why Google is providing the glasses to testers, to get feedback on things like how watches should interact with glasses. 

“I think there’ll be some awesome use cases between them,” Samat said.

What Android XR apps and software will be like

Android XR is designed to span a spectrum of different types of devices, much like the version of Android that runs on phones. So it makes sense that not all apps will look the same. Samat says they’ll generally be broken down into three tiers, the first being two-dimensional apps, which are essentially enlarged versions of the apps already available in the Google Play Store.

Then there are spatialized apps, such as a media app that distributes certain elements of the app, like the comments section on YouTube, for example, in the space around you for easier viewing. Fully immersive apps, on the other hand, would be built specifically with mixed reality in mind. Google Maps in 3D, for instance, which CNET got to try in mixed reality, extends a 3D map landscape out that you can dive into, or flip back into a standard 2D mode.

As is the case with phones, you can expect companies to put their own spin on Android XR depending on their device. Samat says device makers will be able to bring their own services and assistants to Android XR, although some elements of the operating system will remain consistent across devices for ease of use.

There’s a lot at stake for Google, Samsung and Qualcomm when it comes to getting this right. Google has already run into its fair share of snafus with its AI technology, particularly around errors in its AI Overviews tool in search. As devices like the Rabbit R1 and Humane AI Pin have proved, making a first impression is important. People likely won’t be willing to spend their time and money on a gadget they find underwhelming, superfluous and expensive. 

Perhaps the biggest question looming over Google, Samsung and Qualcomm’s mixed-reality endeavors is: In a world in which we’re glued to our phones, is there room for yet another gadget in our lives?

Samat thinks the answer is yes, but it’ll be a gradual shift. Smart glasses won’t replace phones, primarily because they’ll still need to rely on them for some computing power in the near term. But he does think the immediate benefit will become obvious right away, particularly in that smart glasses can make us feel more connected and less distracted.

“It should give you some superpowers in places that you didn’t have them before,” he said. “And you’re like, ‘Wow, this is like, better than my smartphone.'”