The Echo Chamber Attack That Outsmarted GPT-5

Written by

C Sarath Chandra
K. Nanda Kishor Reddy
K. Varun

A few years ago, we thought AI assistants were just chatbots.

Now we’ve got agentic AI systems that decide what to do and then just… go do it. They can write code, order products, move files, run scripts all without asking you every step of the way.

That’s incredible power. And “with great power comes great responsibility”.

So, what’s “agentic AI” in plain English?

Think of it as a co-worker who:

  • Knows your calendar, email, and tools.
  • Can make decisions.
  • Doesn’t always need to ask you first.

Handy? Absolutely.

Safe? Only if you lock it down.

Why security matters here

If an attacker tricks an agentic AI:

  • It’s not just giving bad answers it might do something bad.
  • It could leak sensitive info without realizing.
  • It could even run harmful commands in connected systems.

Once it’s fooled, it’s like handing your house keys to someone wearing a fake delivery badge.

The Echo Chamber Attack on GPT-5

On August 7, 2025, researchers pulled off something wild.

They didn’t just walk up to GPT-5 and say, “Hey, do something bad.”

That wouldn’t work the safety filters would shut it down.

Instead, they played the long game:

  1. Started with harmless prompts random words, simple stories.
  2. Each time, they asked GPT-5 to tweak or expand the story.
  3. Slowly, they wove in hidden instructions, buried in fiction.
  4. By the end, GPT-5 was giving answers it normally never would because it thought it was still “just storytelling.”

This works because the AI remembers the context from earlier in the chat. If you can steer that context step-by-step, you can guide it past its guardrails without ever triggering alarms.

That’s why it’s called the “Echo Chamber” the AI just keeps bouncing back what’s been fed to it, until the tone changes completely.

Other ways agentic AI can be attacked

  • Prompt injection: Hide bad instructions inside a normal request.
  • Context poisoning: Slip in false info so future answers are wrong.
  • Tool misuse: Trick it into using APIs or commands to cause damage.
  • Identity spoofing: Pretend to be someone it trusts.
  • Goal hijacking: Change what it thinks it’s supposed to do.
  • Overload attacks: Spam it until it crashes or slows down.

What can go wrong if it’s not secured

  • Private data leaks.
  • Unwanted purchases or actions.
  • Systems going down.
  • Customers losing trust.
  • Privacy law violations.
  • Long-term model corruption.

How to protect agentic AI

  • Clean and check any data it gets from outside sources.
  • Lock down what it remembers so bad info can’t stick.
  • Only let it use safe tools and APIs.
  • Double-check who’s talking to it.
  • Don’t give it more power than it needs.
  • Keep a human in the loop for high-risk actions.

The takeaway

The GPT-5 Echo Chamber attack shows that the real danger isn’t shouting bad prompts at AI it’s the slow, sneaky stuff.

If you’re building or running agentic AI, your security needs to focus on the conversations before the bad request ever happens.