AI Vendor Risk: The Questions to Ask in Third-Party Due Diligence
Ask vendors where AI is used, what agents can access and change, how your data reaches model providers, and what controls and assurance cover those uses. Use the 15-question shortlist for any vendor, and a deeper, evidence-led review for critical AI services and production agents.
AI is moving faster than the governance and compliance around it. Review most vendors' SOC 2 reports and you'll find system descriptions, risks and controls that don't reflect how much AI the vendor actually uses.
That matters, because when you use a service, you take on its AI risks. Those include the security risks, but also inaccurate outputs, and activities that may be performed in ways that breach your requirements or your privacy obligations.
This guide sets out a short list of questions to ask every vendor, a thorough version for vendors where AI is central to the service, and how to act on the answers.
Why isn't a vendor's SOC 2 or AI certification enough?
Certifications are worth asking about, but none of them shows you where the biggest risks sit in your own use of a vendor's service. AI risk varies so much between use cases that a generic assurance only goes so far:
- ISO 42001 shows a vendor has an AI management system: governance, risk management and oversight. It doesn't show which of its AI uses affect you, or how.
- AIUC-1 is a certification standard for AI agents from the Artificial Intelligence Underwriting Company, grounded in technical testing of agent behavior (AIUC-1 overview). It shows agents are governed and tested, but it's usually scoped to specific products, and the standard is revised often, so check which product and version were certified.
- SOC 2 can cover AI, but AI-specific controls often get lost in the wider control set, and many reports don't mention AI at all (even though they should).
So start by asking vendors directly about their AI use, then use certifications as supporting evidence. The proposed 2027 SOC 2 criteria update would add explicit AI points of focus, but remains an exposure draft. Check which criteria the vendor's report actually uses.
Where does AI risk come from in a vendor's service?
From three places, in falling order of risk:
| Where AI is used | Example | What it exposes you to |
|---|---|---|
| Agents operating in production | An agent that accesses customer data or makes changes in live systems | Unauthorized or incorrect actions on your data, and agents straying from their purpose |
| AI features core to the service | AI that analyzes, generates or decides as part of what you're paying for | Inaccurate outputs you rely on, and your data flowing to model providers |
| AI in the vendor’s operations | AI used in support, development, or the vendor’s own compliance program | Weaker controls behind the service, and compliance work you can't fully rely on |
For worked examples of the controls each pattern needs, see AI controls in practice.
What are the most important AI questions to ask a vendor?
These 15 questions work for any vendor. Send them as a short, standalone request rather than burying them in a long annual questionnaire.
- Is AI used in production, such as agents that can access our data, change production systems or act on our behalf? Describe the extent.
- Which AI features are core to the service we use, and what do they do with our data?
- How is AI used in your own operations, including support, development and your compliance program?
- Is every AI use case and agent individually defined, with a stated purpose, objectives, risks and a named owner?
- Does each agent operate under its own unique identity, so every action it takes can be traced?
- What can each agent access and change, who approved that scope, and how is it enforced?
- Which high-impact actions require human approval before an agent acts?
- Which model providers do you use? Do they train on or retain our data, and where is it processed?
- How are changes to models, prompts and agent configuration assessed, tested and approved before release?
- How do you test the accuracy and reliability of AI outputs, and how are errors caught and corrected?
- How do you protect against prompt injection and agents acting outside their purpose?
- Are agent activities logged? How often are the logs monitored, how many issues have you found, and what did you do about them?
- How will you tell us about AI incidents, or material changes to how AI is used in the service?
- What independent assurance covers your AI use, such as ISO 42001, AIUC-1 or SOC 2, and what does it not cover?
- What can we configure or opt out of, and what are we responsible for on our side?
Which risk areas need a thorough AI due-diligence review?
For critical vendors, or any vendor running agents in production, go deeper. Each area below lists the questions, then what a good answer looks like.
What AI inventory and use should a vendor disclose?
- Can you share an inventory of every AI use case, agent and model in the service and in your operations?
- Which of these touch our data, or act in production on our behalf?
- Do any AI capabilities you market to customers fall outside this inventory?
What good looks like: a maintained register that matches what the vendor markets, with production uses clearly flagged. Vendors sometimes promote AI capabilities to prospects while playing them down in compliance questionnaires, so compare the two.
Who owns AI governance and risk assessments?
- Who owns each AI use case, and who is accountable for AI governance overall?
- Has each use case been risk assessed, and has an impact assessment considered its effects on individuals?
- How often are those assessments reviewed?
- Do you have an AI governance policy, and how is it enforced?
What good looks like: named owners, risk and impact assessments tied to each register entry, and reviews at least annually and whenever a use case changes.
How are agent identity, access and scope controlled?
- Does each agent run under its own identity, separate from human users and other agents?
- How are each agent's permissions limited to what its purpose needs?
- Who can use, configure or change each agent, and how is that access reviewed?
- How are our environment and data isolated from other customers' agents?
- Which actions need human approval, and who gives it?
What good looks like: unique identities, least-privilege scope, a documented authority matrix, isolation between customers, and human approval for high-impact actions.
What should you ask about data and model providers?
- Which model providers, and which models, power each AI feature or agent?
- Do you or your providers use our data to train or improve models?
- How long are prompts, inputs and outputs retained, and where?
- Where is our data processed and stored, including by model providers?
- Are model providers listed as subprocessors and covered by your data processing agreements?
- What happens to the service if a model provider is unavailable?
What good looks like: providers disclosed as subprocessors, contract terms on training and retention, known data locations, and a fallback if a provider goes down. A vendor's model provider is a fourth party to you, which most due diligence doesn't yet cover.
How are model and configuration changes managed?
- How are model upgrades, prompt changes and agent configuration changes assessed before release?
- Do you test changes against defined evaluations before deploying them?
- Is there an impact assessment when a change affects how AI behaves for customers?
- Will you notify us of material changes, such as a new model provider?
What good looks like: AI changes go through change management, with evaluation, approval and tracked model versions, and customers are told about material changes.
How are accuracy and reliability measured?
- How do you measure the accuracy of AI outputs, and what are the current results?
- How do you detect and handle confidently wrong outputs?
- Where do people review outputs before they're relied on?
- How can we report errors, and how are they corrected?
What good looks like: defined evaluations with tracked results, human review where errors matter most, and a clear process for correcting mistakes.
How does the vendor defend against AI-specific attacks?
- How do you defend against prompt injection, including through content agents read?
- How do you limit the tools, data and actions agents can reach, so a compromised agent can't do much harm?
- Has your AI been red-teamed or tested adversarially, and by whom?
- How do you prevent agents exposing one customer's data to another?
What good looks like: layered guardrails, tightly constrained tools, independent adversarial testing, and customer isolation that has been tested, not just designed.
How are agent activities logged, monitored and investigated?
- Are all agent actions logged, with inputs, outputs and approvals, and protected from tampering?
- How often are the logs monitored, and by whom or what?
- How many issues has monitoring found in the past year, and what was done about them?
- How do you detect an agent diverging from its stated purpose?
- What is your AI incident response process, and when will you notify us?
What good looks like: complete, tamper-resistant logs, monitoring at a defined frequency, and a record of issues found and resolved. A vendor that has never found an issue may not be looking closely enough.
What assurance and supporting evidence cover the AI you use?
- Which certifications or reports cover your AI use, and what exactly is in scope?
- For AIUC-1, which products, and which version of the standard, were certified?
- Does your SOC 2 system description and control set cover your AI use?
- If not, when will it?
- Can you share supporting evidence, such as register extracts, evaluation results or monitoring reports?
What good looks like: assurance scoped to the AI you actually rely on, backed by evidence beyond the certificate.
What are your responsibilities when using the AI service?
- Which AI features can we turn off or configure?
- What context or configuration does an agent need from us to work safely?
- What do you expect us to review or approve?
- What do your terms of service say about AI outputs and liability?
What good looks like: clear documentation of what's on your side, settings you control, and terms that state who is responsible for what.
How should you act on the answers?
- Tier vendors by AI exposure. Give vendors with agents in production the full review, vendors with core AI features a focused one, and vendors using AI only in operations a lighter check.
- Ask for evidence, not just answers. Register extracts, logs, change records and evaluation results show whether controls actually operate.
- Write it into the contract. Cover training on your data, retention limits, data location, disclosure of model providers as subprocessors, notice of AI incidents and material changes, and your right to request assurance.
- Configure your side. Turn off features you don't need, and limit agents' scope in your own environment.
- Keep watching. Vendors change their AI use after the questionnaire is answered, so a one-off review goes stale quickly (DeepInspect). Reassess at renewal, and whenever a vendor announces a material AI change.
Related guides
- AI controls in practice: three worked examples connects AI risks to controls, audit tests and evidence.
- SOC 2 controls and criteria for AI agents in production covers the production-agent controls to look for in a report.
- ISO 42001 for AI companies explains AI management system assurance and its limits.
- The signals that matter in assurance and security puts vendor evidence into a broader TPRM decision.
- Getting through enterprise security reviews explains the buyer side of vendor assessment.
Sources
- AIUC-1 Certification, Fin
- Vendor AI Risk Assessment for TPRM, Risk Template, March 2026
- Generative AI Vendor Evaluation and Qualitative Risk Assessment, FS-ISAC
- AI Vendor Due Diligence Questionnaire, DeepInspect
About the author
Paul Wenham
Paul is founder of Throughline. He was a manager at PwC and founded AssuranceLab, the audit practice that supported more than 1,200 technology companies through SOC 2 and ISO audits before its acquisition by Sensiba. He has spent his career building, assessing and auditing technology compliance programs.