A bug within Google Gemini has enabled criminals to exploit the AI system for phishing purposes through invisible text embedded in emails. Despite knowing about this issue since last year, reports indicate that Google hasn’t yet addressed it.

Subtle Phishing Techniques

Criminals are utilizing HTML tricks, such as white text or hidden formatting, to slip unnoticeable text into an email. This hidden content appears harmless when the recipient opens the message; however, Gemini processes everything within, including the concealed elements.

Summary Inclusions and Phishing Risks

If the recipient requests a summary of the email, Gemini unwittingly includes the hidden text in its output. The included text might warn that their Gmail password has been compromised. This message seems to come directly from Gemini itself, making recipients more likely to trust it and follow urgent instructions, such as changing passwords or contacting supposed support numbers.

Since Google’s spam filters typically target suspicious links or attachments, criminals avoid including these in their phishing attempts. By doing so, the messages bypass defensive measures and reach targeted inboxes, enabling redirection to phishing sites without relying on obvious red flags.

Detection Challenges

Detecting such malicious content is technically complex. Some spam filters scrutinize Gemini’s output for urgent warnings, URLs, or phone numbers, flagging the content for further review. Other techniques can remove, neutralize, or ignore hidden text within the body of an email.

Education as a Defense

The most effective defense remains education. Organizations should ensure their employees are trained to be skeptical of any urgent requests for action, even if they seem to come from an AI assistant like Gemini.

AI in Phishing: Polymorphic Campaigns

This isn’t the first instance of leveraging AI in phishing attacks. A technique known as polymorphic phishing incorporates AI to randomize various elements of fraudulent emails—such as sender names, subject lines, and content—helping them evade detection systems trained to identify common patterns.

Ironically, Google has long highlighted Gemini’s role in assisting cybersecurity efforts, especially within the Google Threat Intelligence platform. This AI tool aims to provide users with a deeper understanding of threats and more informed insights into potential attacks.