Start with the approval decision
A polished demo can make an AI product feel nearly decided. The supplier answers questions quickly, the feature looks useful and the monthly price fits. The approval decision still concerns one defined use, its data flow, the people affected and what happens when the product fails.
An AI vendor due diligence checklist should create evidence for that decision. A context-specific record is more useful than a generic vendor score. The same system may be reasonable for drafting internal notes and unsuitable for ranking leads or responding directly to customers.
This guide supports practical commercial investigation before signing, renewing or enabling an embedded AI feature. Privacy, legal, cyber, procurement and sector obligations may require qualified advice. A completed worksheet isn't a compliance determination.
If the delivery model is still open, decide whether to build, buy or use a hybrid approach first. Once a product reaches the shortlist, use the organisation's AI governance policy to identify the owner and approval route. This review supplies the vendor evidence that decision needs.
Define the product use before reviewing the supplier
Write a short description of the business outcome, the people using the product and any customer or staff decision it may influence. Map what information enters, where it goes, what returns, which systems connect and who can see the result. Name the person who reviews the output and the safe fallback when the product is unavailable or wrong.
Include ordinary software that has gained an AI feature. Enabling a meeting summary, lead score or writing assistant can introduce a new model provider, data use or subprocessor even when the wider SaaS contract already exists.
The OAIC guidance on commercially available AI products says product due diligence should test suitability for the intended use. It points organisations to testing, human oversight, privacy and security risks, and access to personal information. It also calls for review across the product lifecycle. That makes the data flow and configured use the unit of assessment, rather than the supplier's reputation alone.
Where a product searches or generates from company material, document the knowledge sources, access boundaries and export route. The guide to preparing business knowledge for reliable AI helps expose internal content and permission problems before they're mistaken for supplier defects.
Separate buyer readiness from supplier evidence
Only the business can provide an internal owner, clean source data and enough staff time to review output. Before requesting documents, name the business owner, approved data, acceptance criteria, review capacity, fallback, operating budget and dependent systems.
Use an AI readiness assessment when these foundations are unclear. It separates work the business must do from evidence the supplier must provide. That distinction protects the review from two errors. A weak internal process can make a suitable product look unreliable. A convincing supplier can make an unready team feel prepared.
Request evidence you can inspect
Ask each question in relation to the defined use. Record the answer beside the artefact that supports it. An unsupported assertion stays open, even when it sounds reassuring.
Product, model and performance
- Request product and model documentation, intended uses, known limitations and release history.
- Ask which model or models serve the feature and whether the supplier can change them without notice.
- Request evaluation methods and results relevant to your data, language, users and failure consequences.
- Ask how a person can review, correct, override or stop an output.
- Confirm what logs are available to the buyer and how long they're retained.
The Australian Government's Guidance for AI Adoption: Foundations asks deployers to obtain evidence of testing from providers and align evaluation with intended use and risk. It connects that evidence to monitoring, data governance and cybersecurity. A growing business needs evidence that applies to the version, configuration and conditions it's considering.
Data, access and infrastructure
- Identify every category of data collected, generated, stored or inferred.
- Confirm whether customer data is used for model training, product improvement or human review.
- Request retention and deletion rules for prompts, files, outputs, logs and backups.
- Identify hosting regions, subprocessors, model providers and cross-border data paths.
- Review identity controls, permissions, encryption, security testing and independent assurance.
- Confirm whether data and records can be exported in a usable format.
ASD's procurement and outsourcing guidelines frame cloud services, managed services and cyber supply chains around shared responsibilities, contractual controls and continuing assurance. Those principles help a commercial buyer ask who controls access, who monitors the service and which party acts during an incident. Specialist cyber review may still be necessary for sensitive or important systems.
Operations, change and support
- Request the incident history relevant to the service and the notification process for a new event.
- Ask who responds, what evidence the buyer receives and how quickly the supplier will communicate.
- Confirm service levels, support routes, maintenance windows and business continuity arrangements.
- Define notice for changes to models, terms, subprocessors, hosting, data use and material features.
- Ask how the supplier supports rollback, suspension, migration and decommissioning.
The NIST AI RMF Playbook Manage guidance extends third-party risk work through documentation, monitoring, incident response, contingency planning and safe decommissioning. The commercial consequence is simple. Pre-contract evidence must support operation and exit, not only selection day.
Read the contract as part of the system
The contract decides whether practical controls survive after approval. Review data ownership and permitted use, confidentiality, retraining, subprocessors, material-change notice, service levels, incident notification, assurance rights, liability allocation, suspension, export, deletion, termination and renewal.
The DTA AI procurement checklist is designed for Australian government buyers and works with government policies, procurement methods and laws. Its agency requirements apply in that public-sector setting. Its questions about ownership, retraining, risk sharing, vendor obligations, testing, monitoring and end of life remain useful prompts for a commercial review.
Check that the agreement preserves access to the evidence and actions your control plan needs. A dashboard promise has limited value if logs aren't retained. An incident plan is weak if the supplier owes no useful notification. An export feature is incomplete if the contract doesn't cover format, timing, deletion and support at exit. Ask qualified advisers to review material privacy, cyber, contract, procurement or sector issues.
Turn vendor claims into acceptance tests
Rewrite broad claims as observable conditions. “Accurate” becomes a threshold for defined tasks using representative cases. “Private” becomes specific rules for collection, access, training, retention and deletion. “Secure” becomes configured permissions, test evidence, incident handling and accountable owners. “Easy to integrate” becomes a complete test of field mapping, errors, retries and rollback.
For every material claim, name the threshold, evidence, reviewer and stop condition. Run normal, boundary, failure and unauthorised cases against the configured product. The full AI workflow pre-launch testing method shows how to turn those conditions into reproducible acceptance evidence.
Certifications, policies and vendor statements all contribute information. As an Off Piste editorial synthesis of the cited guidance, configured-use testing and enforceable terms provide stronger decision evidence because they connect the claim to the buyer's actual use and available controls.
Record gaps and conditions before deciding
Don't average away uncertainty with a universal numeric score. A missing artefact matters differently for an internal drafting aid and a product that influences access to an important service. Record the evidence strength, affected people, failure consequences and enforceable controls beside each gap.
The decision becomes easier to audit when the review keeps the evidence and its consequence together.
Use four outcomes. Approve when evidence is sufficient for the bounded use and ownership is clear. Run a bounded pilot when a controlled test can resolve limited gaps without exposing people or the business to material consequences. Escalate when consequences are material, evidence is incomplete or specialist judgement is needed. Reject when the use cannot be supported by credible evidence and enforceable controls.
Two real dimensions make that choice clearer. Evidence strength increases as the buyer moves from an assertion to inspectable documentation, relevant test results, independent assurance and suitable contract terms. Consequence increases with the effect of failure on people, rights, privacy, security, money and essential operations.
Evidence strength and consequences determine the route
Consequence of failure increases
Escalate
Material consequences with weak or incomplete evidence
Approve with controls
Material consequences with strong evidence and accountable controls
Bounded pilot
Limited consequences with gaps that a controlled test can resolve
Approve
Limited consequences with sufficient evidence and clear ownership
Evidence strength increases
This matrix is an editorial decision model, not an official classification or a substitute for professional advice. The written rationale matters more than the quadrant name.
Plan for change, incidents and exit
Approval remains valid only while the product, terms, configuration, data flow and use stay within the reviewed conditions. Name the live measures, supplier contact, internal incident owner, fallback and authority to pause. Record which changes trigger reassessment and what evidence must arrive before renewal.
Measure the outcomes, exceptions, review effort and cost needed to decide whether an AI workflow should scale, change or stop. If a failure occurs, preserve inputs, outputs, logs, configuration and supplier communication. The live workflow failure diagnostic helps separate model, data, integration, permission and operating faults.
Plan exit while cooperation is easy. Test exportability, confirm the return or deletion of data, identify replacement dependencies and keep a usable manual route. At renewal, request current evidence rather than carrying the first approval forward.
Make the next decision proportionate
A defensible vendor review leaves the business with a defined use, inspectable evidence, named gaps, approval conditions and a dated decision. It also makes uncertainty visible. That record supports approval when the evidence fits the consequences and a bounded pilot when safe testing can resolve a specific gap.
If the use, responsibilities or required evidence remain hard to define, an AI opportunity and risk workshop is the proportionate next step. It should clarify the workflow, data boundary, owner and decision criteria before procurement continues. High-consequence or regulated uses should wait for the relevant legal, privacy, cyber and domain expertise.