Gemini stops making you manually hit ‘Ask about screen’

Isn’t it great when Google’s able to use context to recognize what we were trying to do, and jumps in with a helping hand? Maybe you were about to send an email where you mention an attachment but never actually selected any file — Gmail is happy to prevent that embarrassment by offering a helpful reminder. A few months back we spotted Gemini working on a similar trick, and now it’s starting to enter public testing.

Like Gmail, Gemini handles all sorts of attachments — maybe you want to edit an image, or ask it a question about a video. That extends to screenshots of what you’re actively working on, and to streamline that support, Gemini introduced its “Ask about screen” button.

So far, that’s all well and good. But back in September we identified early work on a new “Screen Context” option that wouldn’t even require you to hit that button and explicitly tell Gemini to start looking at your screen. Instead, the system would attempt to recognize when you were asking Gemini about something on your screen, and then automatically take a screenshot to analyze — no further interaction from you required.

This week, we’re finally starting to see access to this tool roll out, and it has started appearing on some of our devices (Thanks: Diego).

When it first arrives for you, you’ll be presented with that introductory message on the left. Just as we saw when initially covering work on this feature, Google gives you the ability to turn it off if you have privacy concerns.

Leave it on, though, and you can get started right away asking about your screen, and leaving Gemini to capture any screenshots it needs to process. If you’ve also gotten access, give it a try, and let us know how well it worked for you down in the comments. In our early testing, it’s about 50/50 on realizing we were asking about on-screen content, but we imagine your mileage may vary significantly based on individual prompts.