At Google’s I/O 2024 conference, Google made several announcements regarding Google Photos and updates to Generative AI in Search. Here’s a breakdown of what’s new:
Google Photos: Ask Photos with Gemini
Google introduced Ask Photos, a new experimental feature designed to enhance your photo searching experience. Here’s what you can expect:
Intuitive Searching: With Ask Photos, you can search for specific memories or information in a natural way, such as requesting to see the best photo from each national park you’ve visited.
This feature aims to simplify your search process, saving you time scrolling through your gallery.
Question-based Assistance: Ask Photos allows you to ask questions about your life, like where you camped last year or when your vouchers expire. Powered by Gemini’s multimodal capabilities, it can understand the context of your photos to provide relevant answers.
For example, you can inquire about the themes of Lena’s birthday parties, and Ask Photos will analyze details like decorations to give you accurate responses.
Task Assistance: Google Photos now assists you with tasks beyond search, thanks to Gemini models. For instance, after a trip, Ask Photos can help curate the best photos for sharing with friends and family.
It can even suggest top pictures and write personalized captions for social media posts.
How Ask Photos Works
- Understanding Your Query: Ask Photos comprehends your question and conducts a sophisticated search, identifying relevant keywords and natural language concepts to find the answer.
- Crafting the Response: Utilizing Gemini’s capabilities, Ask Photos analyzes photos and crafts helpful responses, ensuring accuracy.
- Safety Measures: While experimental, Ask Photos incorporates safety measures and AI models to provide safe and appropriate responses. It also remembers corrections and additional information provided by users.
Privacy Protection
Google assures users that personal data in Google Photos is never used for ads. Additionally, conversations and personal data in Ask Photos are not reviewed by individuals, except in rare cases to address abuse or harm.
Google emphasizes that generative AI products outside of Google Photos, including other Gemini models and products, are not trained on personal data from Google Photos.
All data in Google Photos is protected with industry-leading security measures.
Availability
Ask Photos is an experimental feature that Google will begin rolling out soon, with plans for further capabilities in the future.
Generative AI in Search
Alongside the announcement of Ask Photos, Google also introduced updates to Generative AI in Search, featuring expanded AI Overviews, enhanced planning and research capabilities, and AI-organized search results. These updates include:
Quick Answers with AI Overviews
Google introduces AI Overviews in Search to provide quick answers without the need to gather information manually. Users can now obtain a brief overview of a topic along with links for further exploration.
Google reports that billions of users have already utilized AI Overviews, leading to increased satisfaction with search results. Additionally, the links included in AI Overviews receive more clicks compared to traditional web listings.
Availability: AI Overviews are rolling out to users in the U.S. this week and will expand to more countries by the end of the year.
Adjusting AI Overviews
Soon, users will have the option to adjust AI Overviews according to their preferences, simplifying language or expanding details. This feature will be particularly beneficial for those new to a topic or seeking simplified explanations.
Availability: This update will soon be available in Search Labs for English queries in the U.S.
Complex Question Support
With the advanced multi-step reasoning capabilities of the custom Gemini model, AI Overviews can assist with complex questions. Users can now ask intricate questions in a single search, eliminating the need for multiple queries.
Availability: Multi-step reasoning capabilities will soon be integrated into AI Overviews in Search Labs for English queries in the U.S.
Planning Assistance
Search will now offer planning capabilities directly within the platform.
Users can receive assistance in creating plans for various purposes, such as meals and vacations. Search will provide personalized recommendations and allow for easy adjustments.
Availability: Meal and trip planning features are currently available in Search Labs for English queries in the U.S., with customization options and additional categories coming later this year.
Explore AI-Organized Results
Google Search will utilize generative AI to brainstorm and create organized results pages, making it easier for users to explore diverse perspectives and content types.
Video Assistance with AI
Users can now ask questions using video, enabling more efficient troubleshooting and assistance for visual issues.
Availability: Video search functionality will soon be available for Search Labs users in English in the U.S., with expansion to more regions in the future.
Google emphasized that these updates represent just a glimpse of their efforts to enhance Google Search by integrating advanced capabilities of the Gemini model. They aim to streamline searching, simplifying, researching, planning, brainstorming, and more.
Users can sign up for Search Labs to be among the first to experience these features.