AI vendor assessment questions to ask before you sign

Before you sign up for an AI tool, send the vendor a short set of written questions about how they use, keep and protect your content. Ask about the exact plan you would buy, and get the answers in writing. The list below covers 9 topics. You can copy it as your AI vendor questionnaire, and it shows you what a good answer and a red flag look like for each one.

Why ask vendors directly

You might wonder why you can’t read the vendor’s privacy page and stop there. There are 2 reasons.

A privacy page is not a contract. It describes what the vendor says it does today. The vendor can usually edit it, and it may not be what you agree to when you sign. What binds the vendor is the contract and any documents it points to, such as a data processing agreement.

Plans differ. Many AI vendors treat free, individual and business plans very differently. A promise about training or data retention on the business plan may not apply to the cheaper plan you were thinking of buying. A general web page often blurs this.

Asking directly fixes both problems. You get an answer about your plan, from a named person, on a date you can point to later. That is the core of AI vendor due diligence for a small organization: you don’t need a security team, you need clear questions and a record of the replies.

If you want to start from what vendors already publish, our tool profiles cover several of these questions for everyday AI tools, based on the vendors’ own pages. Where a vendor does not publish something, we mark it “not documented”. Those gaps are good places to start your own questions.

The questions to ask, grouped by topic

For every group, the same rule applies.

• A good answer is specific, in writing and tied to the plan you would buy.
• A red flag is a link to a general marketing page, an answer that only covers a different plan, or no answer at all.

Below, each group lists the questions to send, then what to look for.

1. Training and use of your content

• Is our content (prompts, files, outputs, recordings) used to train or improve your models, by default, on the plan we would buy?
• Can an admin turn that off for everyone in our organization?
• Does any human at your company review our content, and when?

Good answer: a clear yes or no for your plan, a description of the admin setting that turns training off for all users (see training opt-out), and specific situations when staff may look at content.

Red flag: “we take privacy seriously” with no yes or no, an opt-out that each user has to find and switch on alone, or silence on human review.

2. Retention and deletion

• How long do you keep our content?
• What happens when a user deletes something?
• What happens to our data when we cancel?
• Do you offer zero data retention or shorter retention?

Good answer: a stated time period, an explanation of whether deletion removes content from backups and logs and how long that takes, and a clear process at cancellation.

Red flag: “we keep data as long as necessary” with no period, or no answer about what happens after you leave.

3. Subprocessors and model providers

• Which other companies process our content, including the AI model providers you use?
• Do they train on our content or keep it?

Good answer: a named list of subprocessors, with what each one does, and a statement of whether each can train on or keep your content.

Red flag: a refusal to name the model provider, or a list with no word on training or retention. Many AI tools are built on another company’s model, so this question matters as much as the first group.

4. Data location

• Where is our data stored and processed?
• Can we choose a region?

Good answer: named countries or regions for both storage and processing, and a way to pick one if the plan allows it.

Red flag: an answer about storage only, when processing happens somewhere else, or “globally” with no detail. This matters most if your funders, partners or local law expect data to stay in a certain place.

5. Security and assurance

• Which certifications or audit reports do you have (for example a SOC 2 Type II report, ISO/IEC 27001 or ISO/IEC 42001)?
• Will you share the report?
• Do you support single sign-on and multi-factor authentication?

Good answer: named reports or certificates, an offer to share the report (often under a confidentiality agreement), and yes to multi-factor authentication on your plan.

Red flag: a badge on the website but no report available, or single sign-on offered only on a much more expensive plan with no other protection for logins.

6. Admin controls

• Can we manage users centrally?
• Can we set data settings for everyone at once?
• Can we see audit logs?
• Can we control connections to other apps?

Good answer: a yes to each, with a short description of where the setting lives and which plan includes it.

Red flag: controls that exist only on a plan you aren’t buying, or no way to stop staff from connecting the tool to your email, files or calendar.

7. Incidents and vulnerabilities

• How, and how fast, will you tell us about a security incident affecting our data?
• How do you handle security vulnerabilities in your product?

Good answer: a stated notice period, the channel they use (for example email to a named admin), and a public way for people to report security problems.

Red flag: “we will notify you as required by law” with nothing more, or no process for reports from outside.

8. Contract

• Will you sign a data processing agreement?
• Do the contract terms match the privacy page?
• Can terms change without notice?

Good answer: yes to a data processing agreement, confirmation that the contract covers the promises you care about, and advance notice before terms change.

Red flag: the training or retention promise appears on the website but not in the contract, or terms can change at any time with no notice.

9. AI features specifically

• Can we turn AI features off?
• Can the AI act on our behalf (send, edit, delete), and can we require approval first?

Good answer: an admin switch for AI features, and a clear setting that makes the AI ask a person before it sends, edits or deletes anything.

Red flag: AI features that switch on by themselves with no way to turn them off, or an assistant that can take actions with no approval step.

How to send the questions, record answers and decide

A good AI vendor assessment does not need special software. It needs a written trail.

1. Send the questions in writing. Email is fine. Name the plan you are considering in the first line, and ask the vendor to answer for that plan. A sales call can help, but follow it up in writing.
2. Record each answer with the date and the person who replied. A simple spreadsheet works: one row per question, one column per vendor, plus columns for the date and the name of the person who answered. Paste their words, not your summary.
3. Mark each answer. Use 3 labels: good, unclear, red flag. Anything unclear goes back to the vendor as a follow-up question.
4. Compare at least 2 vendors. Seeing how a second vendor answers shows you what is possible. If one vendor gives specific answers and another sends marketing links, that tells you something.
5. Decide which answers you can’t live without. For most small organizations, the must-haves are no training on your content (or an admin opt-out for everyone), a clear retention period, multi-factor authentication and a signed data processing agreement. Set your own list before the answers arrive, so a good demo doesn’t sway you.
6. Keep the record with the contract. If something changes later, you can show what you were told.
7. Repeat the key questions at renewal. Vendors change plans, providers and terms. Ask the training, retention, subprocessor and contract questions again before you renew.

For a guided walk-through with practice examples, see the course lesson on vendor questions to ask before you sign.

What to watch for

• Answers for the wrong plan. A reply about the enterprise plan does not cover the team plan. Ask again.
• Links instead of answers. A link to a help page is a start, but ask the vendor to confirm in writing that it applies to you.
• Promises outside the contract. If something matters to you, ask for it to appear in the contract or data processing agreement.
• Staff signing up on their own. Your assessment only protects you if people use the plan you checked. Tell staff which account to use.

Frequently asked questions

What if a vendor will not answer?

Treat no answer as a red flag, especially for the training and retention questions. Ask once more, politely and in writing, and note the date. If they still won’t answer, choose another vendor, or keep sensitive content out of that tool.

Do we need all these answers for a free tool?

Free tools often have the weakest terms and no way to sign a contract, so you often can’t get most of these answers. At a minimum, find out whether your content is used for training and how long it is kept. If you can’t, don’t put personal or confidential information into it.

What is a SOC 2 report and should we read it?

A SOC 2 report is an independent auditor’s review of a vendor’s security controls. You don’t need to read every page. Check that the report is recent, that it covers the product you are buying, and whether the auditor noted any problems (often called exceptions). Our SOC 2 glossary entry explains the terms.

How often should we re-ask the questions?

Ask the key questions again at every renewal, and any time the vendor announces new AI features or changes its terms. If you only manage one review a year, focus on training, retention, subprocessors and the contract.