Gemini, Google's AI assistant, is expanding its capabilities on Android phones by rolling out the "Live" feature with camera and screen sharing. This enhancement allows users to have more interactive and context-aware conversations with Gemini, opening up new possibilities for assistance and productivity.
The Gemini Live feature, powered by Project Astra, lets the AI see and understand what's on your phone's screen or what the camera is pointed at. This real-time visual input significantly enhances Gemini's ability to provide relevant and helpful responses.
Currently, the rollout is focused on users of Pixel 9 and Samsung Galaxy S25 devices. Google has officially started rolling out camera and screen sharing in Gemini Live for Android to these users. While initially, there were rumors of exclusivity for flagship devices, the company is planning to extend the availability to more Android devices.
How to Use Gemini Live with Camera and Screen Sharing
Once the feature is available on your device, there are a couple of ways to initiate a Gemini Live session with camera and screen sharing:
- Share Screen: Launch the Gemini overlay and tap the new "Share screen with Live" chip. Android will then ask you to confirm that you want to share your entire screen with the Google app. Notably, the "Share one app" option is currently disabled.
- Share Camera: Open Gemini Live as you normally would and tap the screen share button. You can then switch to the rear camera to show Gemini what you're seeing in the real world. Google recommends capturing objects with steady movements for better results.
Potential Uses and Benefits
The addition of camera and screen sharing to Gemini Live unlocks a range of practical applications:
- Troubleshooting: Point your camera at a malfunctioning appliance, like an internet router, and ask Gemini for troubleshooting advice.
- Organization: Aim your camera at a cluttered space, such as a messy drawer or closet, and ask Gemini for suggestions on how to categorize items and maximize space.
- Creative Brainstorming: Share images or screenshots that inspire you and ask Gemini to help you generate ideas for designs, creative writing, or crafts.
- Personal Shopping: Share your screen while browsing online retailers and ask Gemini for product comparisons, style advice, or recommendations for complementary items.
- Skill Development: Share your screen with Gemini and get feedback on your work, such as blog posts, social media campaigns, or photos.
Gemini Advanced Subscription
While the camera and screen sharing features are currently available for free on Pixel 9 and Galaxy S25 series devices, wider availability on all Android devices will eventually require a Gemini Advanced subscription. This subscription, priced at $19.99 per month, is part of the Google One AI Premium plan and includes other benefits such as Gemini in Gmail and Docs, 2TB of storage, and priority access to new features.
The Future of Gemini and Google Assistant
The rollout of Gemini Live with camera and screen sharing marks a significant step in Google's efforts to create a more intelligent and versatile AI assistant. Over the coming months, Google plans to upgrade more users on mobile devices from Google Assistant to Gemini. Later this year, the classic Google Assistant will no longer be accessible on most mobile devices or available for new downloads in mobile app stores. Google is also bringing a new experience, powered by Gemini, to home devices like speakers, displays, and TVs.
While Gemini is poised to replace Google Assistant as the primary assistant on Android phones, some Google Assistant voice features are not yet available through the Gemini app. Users can switch back to Google Assistant in settings if needed.
