With text, audio, or video inputs, Google’s new AI assistant, Project Astra, provides real-time replies to requests. It locates objects, clarifies code, identifies objects, and offers dog name suggestions. Through the Gemini Live interface, Astra’s capabilities will be incorporated into the Gemini app.
Google unveiled Project Astra, a new multimodal AI assistant that can respond to users’ questions in real-time using text, voice, or video inputs. The new AI assistant can be seen interacting with the user and providing real-time answers in a sample video that Google provided. This feature is similar to what OpenAI demonstrated a day ago with its GPT-4o model.
Google’s new AI assistant could locate the user’s glasses, detect objects in the room, recognize and explain a certain portion of the code, locate it precisely by gazing out the window, and even come up with imaginative names for dogs. Furthermore, Google demonstrated how to integrate Project Astra with smart glasses or a smartphone, indicating that Google Lens may get a significant Gemini-powered update in the future.
According to Google, Project Astra can handle data more quickly because it encodes video frames, compiles audio and video input into an event chronology, and caches all of this data for later use. Additionally, Google has improved the new AI assistant’s audio quality by adding more natural-sounding voices and allowing users to choose between them.
“To be truly useful, an agent needs to understand and respond to the complex and dynamic world just like people do — and take in and remember what it sees and hears to understand the context and take action,” stated Demis Hassabis, CEO of Google DeepMind, in explaining the necessity for an autonomous AI agent. For consumers to converse with it organically and without lag or delay, it must also be proactive, teachable, and personable.”
Later this year, the Gemini app and other Google products will be able to access Project Astra’s capabilities through the Gemini Live interface.
