Twins can now answer questions about what’s on your smartphone screen
Gemini for Android is getting two new features that let users ask questions about their smartphone’s current screen. Called “Ask About This Screen” and “Ask About This Video,” these tools take a screenshot and parse the contents of the screen. When the user asks, it can respond with relevant information the user is looking for. The video feature, which appears to be compatible only with YouTube, can also provide video summaries, provided that subtitles have been added to the video.
Gemini now lets users ask questions about their screen
Gadgets 360 staff spotted the features in Google app version 15.33.38.28.arm64 and later on several smartphones. They appear at the top of the floating Gemini window when a user invokes the AI assistant. Those who have access to the feature will see a separate rectangular strip with the text “Ask about this screen” and a screenshot icon. If Gemini was activated while playing a video on YouTube, the text is replaced with “Ask about this video” and the YouTube icon.
Gemini Ask about this screen feature
Tapping the “Ask about this screen” icon will capture a temporary screenshot in Gemini, and the user can then ask any question about the screen. The answers will be based solely on what the AI can see in the screenshot, but users can also ask Gemini to go online to find more information. One use case could be asking for a short summary of a news article.
To ask a follow-up question, users can simply tap on the microphone icon or the keyboard icon at the bottom of the floating window. It is notable that the captured screenshot is not saved on the device. Also, long screenshots are not supported with this feature.
The Ask About This Video feature only works for a YouTube video that has subtitles added. During our testing, we found that no other video platform, despite its status as subtitles, was compatible with the feature. However, for a YouTube video, users can ask questions about the video using the feature. The answers are generated based on the subtitles and the video itself is not analyzed.
