Browser Extensions Can Exploit ChatGPT, Gemini in ‘Man in the Prompt’ Attack

A new cyberattack method, dubbed Man in the Prompt, has been identified, allowing malicious actors to exploit common browser extensions to inject harmful instructions into leading generative AI tools like ChatGPT, Google Gemini, and others. This critical finding comes from a recent threat intelligence report by cybersecurity research firm LayerX.

According to researchers, it all starts with how most AI tools function within web browsers. Their prompt input fields are part of the web page’s structure (known as the Document Object Model, or DOM). This means that virtually any browser extension with basic scripting access to the DOM can read or alter what users type into AI prompts, even without requiring special permissions.

Bad actors can use compromised or outright malicious extensions to carry out various harmful activities. These include manipulating a user’s input to the AI, secretly injecting hidden instructions, extracting sensitive data from AI responses or the entire session, and even tricking the AI model into revealing confidential information or performing unintended actions. Essentially, the browser becomes a conduit, allowing the extension to act as a “man in the middle” for AI interactions.

The risk is significant because browser-based AI tools often process sensitive information. Users may paste confidential company data into these interfaces, and some internal AI applications trained on proprietary datasets can be exposed if browser extensions interfere with or extract content from the prompt or response fields.

The ubiquity of browser extensions, coupled with the fact that many organisations allow free installation, means a single vulnerable extension can provide an attacker with a silent pathway to steal valuable corporate knowledge.

“The exploit has been tested on all top commercial LLMs, with proof-of-concept demos provided for ChatGPT and Google Gemini. The implication for organisations is that as they grow increasingly reliant on AI tools, that these LLMs, especially those trained with confidential company information, can be turned into ‘hacking copilots’ to steal sensitive corporate information.”

LayerX

LayerX demonstrated proof-of-concept attacks against major platforms. For ChatGPT, an extension with minimal declared permissions could inject a prompt, extract the AI’s response, and remove chat history from the user’s view to reduce detection.

For Google Gemini, the attack exploited its integration with Google Workspace. Even when the Gemini sidebar was closed, a compromised extension could inject prompts to access and exfiltrate sensitive user data, including emails, contacts, file contents, and shared folders.

Google was informed about this specific browser extension vulnerability by LayerX. Check out this exploit’s demo here:

This attack creates a blind spot for traditional security tools like endpoint Data Loss Prevention (DLP) systems or Secure Web Gateways, as they lack visibility into these DOM-level interactions. Blocking AI tools by URL alone also won’t protect internal AI deployments.

LayerX advises organisations to adjust their security strategies towards inspecting in-browser behaviour. Key recommendations include monitoring DOM interactions within AI tools to detect suspicious activity, blocking risky extensions based on their behaviour rather than just their listed permissions, and actively preventing prompt tampering and data exfiltration in real-time at the browser layer.

Mayank Kumar, Founding AI Engineer at DeepTempo, highlighted the broader implications of this new attack vector in his comment shared with Hackread.com. “The pressure to integrate generative AI is real,” he observed, noting that organisations widely adopt models like ChatGPT and Gemini for productivity gains. However, he warned, this rapid adoption is “severely testing the security infrastructure built in the pre-GenAI era.”

Kumar emphasised that attacks like “Man in the Prompt” highlight the critical need to rethink security for the interfaces where proprietary data, AI tools, and third-party integrations like browser extensions interact. He stated, “Prompts are not just text, they are interfaces.” This new reality means securing not just the AI model, but the entire data journey through potentially vulnerable browser environments.

Kumar advocates for going “beyond surface-level protection” by implementing deep-layer network monitoring. By looking for anomalies in network traffic correlated with AI tool interactions, organisations can detect suspicious activities, such as unusual data leaving the network or unexpected communications, even when hidden within seemingly legitimate AI prompts. This layered approach, combining application awareness with strict network scrutiny, is vital to counter this new wave of AI-driven cyber threats.