AI Vendor Risk Assessment: What to Ask Before You Buy an AI Tool

An AI vendor risk assessment checks what a supplier’s model does with your data before you sign. The questions to ask and the evidence to demand.

By Krikor Tengerian · Co-founder, SecuritAI Technologies Ltd. · October 2026

Your operations lead wants to switch on an AI assistant inside a tool you already pay for. The vendor sent a link to their trust page and a SOC 2 report, and procurement is asking you to sign off by Friday.

That is how most Canadian companies meet this problem. Nobody set out to buy an AI system. A feature arrived inside a product you already use, or a sales rep offered one, and suddenly a model you have never seen is reading customer emails. It is not a sign your process is broken. It is a sign your old vendor questionnaire was written before the question existed.

Here is the part people get wrong

People assume the vendor’s existing security paperwork covers the AI. It usually does not. A SOC 2 report describes the controls a vendor runs over its own systems, in the scope the vendor chose. It was not designed to tell you what a model does with your prompts, whether your data trains the next version, or whether anyone tested the feature against prompt injection before it shipped. Unless the vendor put those things in scope, the report is silent on them.

The second mistake is thinking the risk moved to the vendor when the contract was signed. It did not. The Office of the Privacy Commissioner of Canada states it plainly in its guidance on processing personal information by a third party: organizations are responsible for personal information in their possession or custody, including information transferred to a third party for processing. A customer record pasted into a vendor’s AI assistant is still your customer record.

So the gap is rarely the law. It is evidence. When a buyer or a regulator asks what that AI tool does with your data, most teams can produce a sales deck and nothing else.

What Canadian buyers already have to work with

There is no AI specific federal law in force in Canada, which we cover in our guide to AI governance in Canada. You do not need one to run a sound assessment. Four public sources give you the structure, and each one is free to read.

The NIST AI Risk Management Framework. The NIST AI RMF Playbook has a subcategory written for exactly this. GOVERN 6.1 calls for policies and procedures that address AI risks associated with third-party entities. GOVERN 6.2 calls for contingency processes to handle failures or incidents in third-party data or AI systems deemed to be high risk. Among its suggested actions is thorough testing of third-party AI systems. It is voluntary, and enterprise buyers often name it when they ask about your own AI.

OSFI, if you sell to banks or insurers. Guideline B-10 on third-party risk management applies to federally regulated financial institutions. It expects them to assess each third-party arrangement for its risk and criticality, and to scale the depth of that work to the risk. Guideline E-23 on model risk management, effective May 1, 2027, names the surge in AI and machine learning models directly and says the framework covers models or data sourced from third-party vendors. If your customer is a bank, your AI supplier is about to become their question too.

The federal generative AI guide. The Government of Canada’s guide on the use of generative artificial intelligence tells federal institutions to have legal, security and privacy teams review a supplier’s terms of use and privacy policy before using any system to process sensitive or proprietary information. It is written for government, and it is a sensible floor for anyone.

OWASP. The OWASP Top 10 for LLM Applications 2025 lists supply chain as its own risk, LLM03, and warns that LLM supply chains are susceptible to vulnerabilities affecting the integrity of training data, models and deployment platforms. That is the technical reason the model underneath a product matters as much as the product.

The AI vendor risk assessment questions to ask

Group the questions so the vendor cannot answer one section with another. For each one, ask for evidence, not a yes.

Your data

• Where are prompts, files and outputs stored, in which country, and for how long?
• Is any of our data used to train or improve a model, including the vendor’s model provider’s? Where is the opt out, and is it on by default?
• Which subprocessors see our data, and does that list include the company that built the underlying model?
• How do we delete our data, and what proof do we get that it happened?

The model

• Which model does the feature run on, whose is it, and which version?
• How will we be told when the model or its provider changes? A version change can alter behavior without any change to the product you bought.
• What instructions sit around the model, and can a user’s input override them?

Testing

• Has the feature been tested against prompt injection, data leakage and unsafe output before release? By whom, and when was the last test?
• Can we see a summary of the findings and what was fixed?
• Does the vendor test again after a model change, or only at launch?

Access and logging

• Who at the vendor can read our prompts and outputs, and is that access logged?
• Can we export a log of what our users sent and what the model returned?
• Can the feature act on its own, such as sending email or changing records, and can we turn that off?

When it fails

• What is the incident process for an AI specific failure, such as the model disclosing another customer’s data?
• How fast will we be told, and through what channel?
• If the feature is withdrawn or the model provider cuts off access, what keeps working? This is the contingency question NIST puts in GOVERN 6.2.

The contract

• Can the vendor change the terms of use without telling you? The federal guide has legal teams review a supplier’s terms before any sensitive use, and that review only holds while the terms stay the same.
• Does the contract restrict training on our data in writing, not only on a trust page?
• What happens to our data on exit?

Scale the effort to the risk

Not every AI feature deserves the full list. A grammar checker that never sees customer data is not the same as a support bot answering your clients, and OSFI’s own guidance says the depth of third-party review should be proportionate to risk and criticality.

A workable way to sort vendors is by two questions. Does the feature see personal or confidential information? Does it act, or only suggest? A feature that sees sensitive data and takes actions gets every question above and a test of its own. A feature that sees nothing sensitive and only drafts text for a person to review gets the data and contract sections. Write the tier down, with the reason, so the next reviewer does not redo the argument.

Record it where you can find it again

An assessment that lives in one person’s inbox is gone the day that person leaves. Every AI vendor you approve belongs in your AI system register, with the owner, the tier, the date of the assessment and the model version it was based on. That last field is the one that pays off. When the vendor announces a new model, you know which assessments are now out of date.

The same evidence carries into your own sales cycle. When your customers send you a security questionnaire, the AI section is increasingly about the tools you use, not only the product you sell. If your team answers those questionnaires often, SecuritComply has a practical piece on how Canadian vendors answer security questionnaires faster.

When the answers are not good enough

Some vendors will answer every question well. Many will answer some and deflect the rest. A deflection is information. Three responses should slow a purchase down:

1. The vendor cannot name the model underneath the feature.

2. Training on your data is controlled only by a setting, with nothing in the contract.

3. Nobody has tested the feature for prompt injection, and there is no plan to.

None of these has to end the deal. Each one should end up as a written condition, a narrower rollout, or a feature that stays switched off until the answer improves.

Being straight about our role

We are not an auditor and we do not certify vendors. Nobody can hand you a certificate that says an AI tool is safe. What we build at SecuritAI is software that tests AI systems for attacks like prompt injection and data leakage, and a firewall that sits in front of a model to control what goes in and what comes out. The questions above work whether or not you ever use either one.

If you want to know where your own company stands on AI risk before a buyer asks, the AI security readiness check walks through it.

References

1. NIST, AI RMF Playbook, GOVERN function (GOVERN 6.1 and 6.2)
2. Office of the Privacy Commissioner of Canada, guidance on processing personal information by a third party
3. Office of the Privacy Commissioner of Canada, privacy and outsourcing for businesses
4. OSFI, Guideline B-10, Third-Party Risk Management
5. OSFI, Guideline E-23, Model Risk Management (2027)
6. Government of Canada, Guide on the use of generative artificial intelligence
7. OWASP, Top 10 for LLM Applications 2025, LLM03 Supply Chain

← Back to the blog