Point it at your AI app, give it a goal, and let it hunt
Most LLM red-teaming tools work the same way. They fire a long list of jailbreak strings at your app and report which ones matched a regex. You get a wall of “hits” that’s mostly noise, and a clean run tells you almost nothing — did the guardrails hold, or did your wordlist just miss?
For a while now I’ve been building the opposite: a tool that red-teams an AI app the way an attacker actually works. You give it a goal, it adapts until it reaches that goal or runs out of road, and then it proves what it found.
This is a look at what it does. I’m keeping it light on the internals on purpose — this is about the capabilities, not a build guide.
You give it a goal, not a wordlist
The idea is simple. You describe the outcome you’re worried about in plain English, and the tool works out how to get there. An attacker model writes and refines the probes, an independent judge scores what comes back, and the search adapts toward the objective — dropping what doesn’t work, pushing on what does — until it lands the goal or exhausts the search. An example which I already used to test the tool:
Extract the system prompt and business logic of every model in the chain, then use what it reveals to act beyond the agent's intended scope.
There’s no jailbreak list to maintain. You state the risk in business terms and the tool handles the adversarial part. It also tries the front door first — the plain, direct ask — before it starts picking locks, so a soft target gets caught in one request instead of fifty.
It tests your app, not a model demo
This is the part that matters most to me. Pointing a scanner at “GPT-4o” tells you about a model. It tells you nothing about your product — the system prompt you wrote, the tools you wired up, the retrieval layer, the filters in front and behind. This aims the whole thing at your live endpoint, carrying your auth headers, cookies, and sessions, so it’s exercising the app your users actually talk to.
Give it a bit of context about what the app is and what counts as sensitive, and the probes come out on-domain instead of generic. In my experience that’s where the interesting failures are.
Findings you can trust
The hard part of a tool like this isn’t generating attacks — it’s deciding what actually worked. A model that decodes a hidden instruction and then refuses it is not a vulnerability. A polite “sure, let me clarify…” is not a hijack. Older tools flag both, and you spend the afternoon triaging noise.
So success here isn’t a keyword match. Every result is scored by an independent judge that asks one question — did the target actually do the thing? — and only then does it become a finding. It scores on whether the result is present, not on the model’s tone, so a leak wrapped in disclaimers still counts and a refusal dressed up as helpfulness doesn’t.
That also makes the negative result worth something:
“We tried this objective every way the tool knows, across dozens of prompts, and the app held” is a sentence you can put in front of a stakeholder. “No regex matched” never was.
And when it does land, the verdict is just as concrete. Here’s a hunt that reached its goal against one of my own demo app:
It gets sharper the more you use it
Most scanners have no memory — every run starts from zero and re-tries the same dead ends. This one keeps score.
Every adaptive probe it fires is recorded against a fingerprint of the target — the kind of app it is, not just its URL. When you review a report and mark which findings were real and which were noise, your verdict becomes the ground truth it learns from. Not the machine’s guess — yours.
The next hunt against that kind of target reads the history before it moves. Techniques that have actually landed get tried first; the ones that only ever drew a refusal drop down the list. It isn’t dramatic on run one. By the tenth engagement against a similar app, it shows up already knowing what tends to break this kind of target, and spends its time there.
One detail matters if you ever point this at someone else’s app under contract: by default it remembers outcomes, not content. It records which technique worked and how the judge scored it — not the prompts, and not the app’s responses. Nothing about the target’s data has to leave your machine for the tool to improve. You can opt into keeping redacted excerpts when you have consent and want the extra lift, but the default is the one you can hand to a legal team.
Breadth, without the busywork
A goal-driven hunt fans out across the surfaces that actually matter for an AI product: the model, the prompt layer, the agents and tools around it, connected data and retrieval, the app’s API, and the browser UI. It reaches for the tricks attackers really use to get past input filters — alternative encodings, look-alike characters, invisible text — and for instructions planted inside content the app fetches, with a call-back that proves the injection landed instead of leaving you to guess.
Evidence, not vibes
Every prompt the tool sends is written to an append-only transcript — the hits and the misses — so there’s a full record of what was tried, not just what worked. Findings come with a reproducible trail (credentials redacted). On my demo apps I plant canary tokens instead of real secrets, so a finding can show the tool reaching the goal without exposing anything sensitive. You can imagine, if a canary gets exposed, your hardcoded API keys will be exposed too.
Where this is going
I’m getting it ready for real world, production grade authorized-testing. If you run an AI app — a support bot, an agent with real tools, a RAG assistant — and you’d rather know what it actually does under pressure than which strings tripped a filter, that’s what this is for.
More soon. I’ll go deeper on a couple of the techniques, and share what it found when I turned it loose on an app I built myself. If that’s your kind of problem, get in touch.