Voice User Interface · Android & iOS Design
Gemini Home Extension allows bulk actions in Gemini app across your home devices.
What are Gemini Extensions?
Extensions are opt-ins within Gemini for users to give access to tools to use their data. For example, opting into the YouTube Music extension will allow it to search for songs and playlists based on music and playlists you already have used within YouTube Music.
Google Home Platform brings together all of your smart home devices into one unified experience
We've since brought the experience wherever you are, across different surfaces & modalities
With the Google Home Extension, Gemini makes your smart home even more personal, intelligent & easy to use
Hey Google, set the mood in the living room for Valentines Day ❤️
Key Features
Control your home in new & more helpful ways
Create automations that improve your quality of life
Search your home for insights & memories
Anatomy of Extension Framework
Tap to toggle
Drag to adjust
Long press to open app
Access Deeper Controls
Users can long press on any device to open the dedicated controller in the Google Home App.
2 Factor Auth
For device requiring 2 Factor Auth, users will be directed to Google Home App for code entry.
Entry Points
From lock screen
Say "Hey Google" to a resting phone
Floaty
Quicly long press + pull in screen context with query
In-app
Control your home in Gemini app
Bringing Home Features into Gemini App
Problem
Noisy Events
Motion event detection is particularly noisy today. So we sought a way to use LLM's to search home camera footage for us and only surface what's needed.
Searchable Home
Singular events
Search camera footage for individual events
Scaled events
Search camera footage for scaled summaries
Bardy Queries
Interperative queries
Interpret user intent across smart home devices
Compound queries
Allow users to layer query definition and understand home structures across devices
Media Casting to Home Hardware
Play media on a specific device
Allow users to ask Gemini app to play and control media on their speakers and displays in their home, compounding 2+ extensions in a single query.
Scaling Home Actions to Gemini on Watch
A glimpse at the active feature work of bringing Gemini to Wearables
Mobile flow
Wear flow
