AI penetration testing vs classic pentesting: what actually changes

Why model behaviour cannot be patched shut the way a code flaw can, what an AI test actually covers, and what a useful report has to contain.

By Krikor Tengerian · Co-founder, SecuritAI Technologies Ltd. · August 2026

That difference sounds academic until you try to write the remediation section of the report. Then it decides everything.

Why this is turning up in procurement

Your company shipped a feature with a model behind it. Maybe a support assistant, maybe a document summariser, maybe something that drafts replies for a human to approve. Some months later a security questionnaire arrives from an enterprise buyer, and one line asks whether the AI components have been penetration tested.

The team’s first instinct is to forward last year’s pentest report. That report covered the web application the model now sits inside, and it is a perfectly good report. It also did not test a single thing that makes the model risky, and a reviewer who knows the difference will see that in about a minute.

What classic pentesting does well

A classic penetration test looks for defects in things people built on purpose. Injection flaws, broken access control, misconfigured storage, weak session handling, an unpatched dependency, a server that answers a request it should refuse.

The defining property is determinism. If a tester finds that a request to one endpoint returns another customer’s record, that finding is reproducible. You can write it up, fix the code, retest it, and prove the same request now fails. The bug had a location and the fix has a diff.

That property is what the whole discipline is built on, including the retest culture, the severity ratings, and the expectation that a finding is either open or closed.

The thing that breaks the old model

You cannot patch behaviour shut the way you patch code shut.

Prompt injection is the clearest case. The OWASP Gen AI Security Project’s entry for LLM01:2025 Prompt Injection defines the vulnerability as one that occurs when user prompts alter the LLM’s behavior or output in unintended ways, and splits it into direct injections, where the user’s own input changes behaviour, and indirect ones, where the model ingests external content such as a website or a file that carries hidden instructions.

Then the same page says the part that reorganises how you think about testing. Given, in its words, the stochastic influence at the heart of the way models work, it is unclear if there are fool-proof methods of prevention, and it notes that measures such as retrieval augmented generation and fine tuning reduce the problem without fully removing it.

Sit with that for a moment, because it has three consequences that a classic pentest report has no way to express.

• There is no closed state. A prompt injection finding does not get fixed and retested to zero the way an access control bug does. It gets mitigated, and the honest report says how much and under what conditions.
• A single successful attack is weak evidence, and so is a single failure. If a jailbreak works one time in fifty, running it once and getting a refusal proves very little. Testing has to run enough attempts to say something about the rate, not the existence.
• The attack surface includes content the model reads. Any document, web page, email or ticket that reaches the model is a possible instruction channel. That is not a place classic pentesting has ever had to look.

What AI penetration testing actually covers

The working reference most testers now use is the OWASP Gen AI Security Project’s Top 10 for LLM Applications, whose 2025 list runs from LLM01 Prompt Injection through LLM02 Sensitive Information Disclosure, LLM03 Supply Chain, LLM04 Data and Model Poisoning, LLM05 Improper Output Handling, LLM06 Excessive Agency, LLM07 System Prompt Leakage, LLM08 Vector and Embedding Weaknesses, LLM09 Misinformation, to LLM10 Unbounded Consumption.

Read that list next to a classic pentest scope and the overlap is smaller than people assume. Supply chain and improper output handling have close classic cousins. The rest are new problems that live in the model, its context, or the permissions it was handed.

Two of them cause most of the real damage in practice.

Excessive agency is what happens when the model can do things, not just say things. If the assistant can call an API, send an email, run a query, or move a file, then a successful injection is no longer a rude answer. It is an action taken with your service account’s permissions. The interesting question in a modern test is rarely what can I make it say. It is what can I make it do, and as whom.

Sensitive information disclosure covers the model repeating what it should not, whether that came from its context window, a retrieved document, or another user’s session. This is where a test of the retrieval layer matters as much as a test of the model, because a vector store that returns any document to any user makes the model’s caution irrelevant.

The system is bigger than the model

The most common scoping mistake is treating the model as the target. The model is one component inside a system, and most findings worth reporting live in the joins.

A useful test covers the system prompt and whether it leaks, the retrieval layer and whether it enforces the same permissions as the application, every tool or function the model can invoke and what those run as, the output path and whether downstream code trusts model output enough to render or execute it, and the logging, meaning whether you could reconstruct what happened after an incident.

That last one is a compliance question wearing a security costume, and it is the one teams fail most often. Being able to show a customer which prompts and outputs were retained, for how long, and who could read them is a question that arrives from privacy reviewers, not just attackers.

What a useful report looks like

Because there is no closed state, the report has to carry information a classic one does not.

For each finding it should give the attack, the success rate across a stated number of attempts, the conditions under which it worked, the mitigation applied, and the residual rate after that mitigation. A finding written as open or closed, with no rate and no attempt count, is a classic pentest template stretched over a problem it does not fit.

It should also separate what the model did from what the system allowed. Those get different owners and different fixes. One is a guardrail question, the other is a permissions question, and conflating them sends the wrong team after the wrong work.

Where the frameworks fit

Buyers rarely ask for this by name. They ask through whichever framework they already trust.

The National Institute of Standards and Technology describes its AI Risk Management Framework, released on January 26, 2023, as a framework to better manage risks to individuals, organizations, and society associated with artificial intelligence, and states it is intended for voluntary use. Voluntary matters. Nobody certifies you against it, so what you are asked for is evidence organised the way it organises things, which is exactly what a properly written AI test report gives you. Our practical guide to the NIST AI RMF for Canadian companies covers that mapping in full.

Underneath that sits the law you already had. The Office of the Privacy Commissioner of Canada explains that PIPEDA applies to private sector organizations that collect, use or disclose personal information in the course of a commercial activity, and sets out ten fair information principles including safeguards. A model that quietly retains customer data is a privacy problem before it is ever an AI problem.

If your buyer is also asking for a SOC 2 or ISO 27001 report alongside the AI questions, that is the same evidence problem in a different vocabulary, and SecuritComply covers that side for Canadian companies.

Being straight about the limits

We are not an auditor and we do not issue certificates or attestation reports. Nobody certifies an AI system as safe, and any vendor offering to is selling something the frameworks do not currently support.

What we build is tooling. AI red teaming to run the attacks at a volume that produces rates rather than anecdotes, and an AI firewall to enforce the guardrails once you know which ones you need. The testing tells you where the model bends. The runtime control is what stops it bending in production.

References

1. OWASP Gen AI Security Project, LLM01:2025 Prompt Injection
2. OWASP Gen AI Security Project, OWASP Top 10 for LLM Applications
3. National Institute of Standards and Technology, AI Risk Management Framework
4. Office of the Privacy Commissioner of Canada, PIPEDA in brief

← Back to the blog