At its recent I/O 2024 conference, Google announced a series of updates aimed at integrating Google AI into the Android operating system. These updates promise to revolutionize how users interact with their Android devices. Here’s what Google announced:
Circle to Search for Homework Help
Google introduced an enhanced version of Circle to Search, designed to aid students with homework.
By circling a problematic prompt on their digital materials, students can now receive step-by-step instructions for solving math and physics word problems directly on their phones or tablets.
The feature is already available on over 100 million devices, with plans to double this figure by year-end.
Gemini’s Contextual Assistance
Gemini, Google’s AI, will soon offer contextual assistance by overlaying its features on top of other apps.
Users can drag and drop images generated by Gemini into apps like Gmail and Google Messages or utilize the “Ask this video” function to find specific information within YouTube videos.
This update will reach hundreds of millions of devices in the coming months, with ongoing improvements to provide dynamic suggestions based on on-screen content.
Multimodal Capabilities for Gemini Nano
Google announced the integration of full multimodal capabilities into Gemini Nano, Android’s built-in, on-device foundation model.
This update, set to debut on Pixel devices later this year, enables smartphones to understand various inputs beyond text, including images, sounds, and spoken language.
Enhanced Descriptions with TalkBack
Gemini Nano’s multimodal capabilities will extend to TalkBack, benefiting users with blindness or low vision.
This feature will provide detailed descriptions of images, offering insights into photos shared by family and friends or product details while online shopping.
These descriptions are generated on-device, ensuring swift performance even without a network connection.
Real-time Scam Alerts
Google is testing a new feature that utilizes Gemini Nano to detect conversation patterns associated with scams during phone calls.
Users will receive real-time alerts if the system identifies suspicious behavior, such as urgent fund transfers or requests for personal information like PINs and passwords. This protection occurs on-device, preserving user privacy.
Looking Ahead
Google emphasized that these updates represent just the beginning of integrating on-device AI into the Android experience.
The company plans to expand Google AI across various smartphone brands, including Pixel and Samsung, to further enhance user experiences.
Announcing the updates, Sameer Samat, President, Android Ecosystem, said:
We’re experiencing a once-in-a-generation moment where the latest strides in AI are reshaping the capabilities of smartphones. With Google AI integrated into Android’s core, the billions of Android users now have the opportunity to engage with their devices in entirely fresh ways.
Today, we unveil updates that bring the power of Google AI directly to your Android experience. This is just the beginning of how on-device AI can redefine the potential of your phone, and we’re committed to embedding Google AI into every facet of the smartphone journey.