Recent reports and demonstrations from the Black Hat computer-security conference have shown how outside Gemini AI prompts, dubbed promptware, could fool the AI and force it to control Google Home-connected smart devices. That’s an issue for Google, which has been working to add Gemini features to its Google Home app and replace Google Assistant with the new AI helper.
The secret to these serious vulnerabilities is how Gemini is designed to respond to basic commands in English. Demonstrations show how a prompt sneakily added to an inserted Google Calendar invite will be read by Gemini the same way it scans other Google app data, such as when it is summarizing emails. But in this case, the addition gives Gemini a very specific order, like creating an agent to control everyday devices from Google Home.
The Tel Aviv University researchers, including Ben Nassi, Stav Cohen and Or Yair, have created their own website that showcases their report titled Invitation is All You Need. It includes videos showing how the right Gemini prompts could be used to open windows, turn off lights, turn on a boiler or geolocate the current user.
As the Invitation is All You Need research shows, a detailed prompt can be hidden in an innocuous Calendar invite title or similar spot. These commands can make Gemini create a hidden agent and wait for a common response (like saying “thank you” in an email) to trigger certain actions.
Even if your calendar controls are tight, some of these promptware attacks could be performed through other things that Gemini scans, such as an email subject line. Other demonstrations showed how similar commands could lead to spam messages, deleted events, automatic Zoom streaming and more unpleasant tricks.
Should you worry about your Google Home devices?
Google told CNET they have introduced multiple fixes to address the promptware vulnerabilities since the researchers provided Google with their report in February 2015. That’s the point of the Black Hat conferences — to uncover problems before real cybercriminals seize them, and get the fixes in fast.
Andy Wen, senior director of security product management at Google Workspace, told CNET, “We fixed this issue before it could be exploited thanks to the great work and responsible disclosure by Ben Nassi and team. Their research helped us better understand novel attack pathways, and accelerated our work to deploy new, cutting edge defenses which are now in place protecting users.”
If you’re still concerned, you can disable Gemini entirely in most cases.
As I’ve covered before, smart home hacking is very rare and very difficult with today’s latest security measures. But as these new generative AIs get added to smart homes (the slowly rolling out Alexa Plus and eventual Siri AI upgrades included), there’s a chance they could bring new vulnerabilities with them. Now, we’re seeing how that actually works, and I’d like these AI features to get another security pass, ASAP.
Read the full article here
