Key findings

• Define mandatory outcomes and controls before accepting vendor demos.

• Request evidence for security, privacy, quality, and operational claims.

• Test the real integration with representative and adversarial cases.

• Score serious failures separately from average quality.

• Put monitoring, change notice, incident support, data return, and deletion into the contract.

Prepare a buyer brief before speaking to vendors

Write a two-page brief covering the current workflow, users, case volume, systems, data classes, geographic scope, baseline performance, target outcome, unacceptable outcomes, and budget model. Separate mandatory requirements from preferences. State which actions may be automated, which require human approval, and which are prohibited. Vendors should respond to the business problem rather than substitute a polished generic use case.

Assign reviewers from the process, technology, security, privacy, finance, and legal functions in proportion to risk. For a small purchase, one person may hold several roles, but each lens still needs an explicit answer. Define scoring and disqualification rules before the shortlist. A missing subprocessor list or inability to restrict write permissions should not be rescued by an attractive user interface.

• Evidence to prepare: process map, data inventory, sample cases, target metrics, permission needs, and expected contract term.

• Disqualifiers to define: prohibited data use, missing deletion process, wildcard access, no audit trail, or no incident commitment.

Check business fit and architecture

Ask the vendor to diagram the exact production architecture: user interface, orchestration, model providers, retrieval stores, connectors, execution services, hosting regions, logs, and subprocessors. Identify which components are controlled by the vendor and which are customer-configured. Confirm tenancy isolation, availability dependencies, rate limits, context limits, and the behavior when a provider or integration fails.

Require a walkthrough of the target workflow from trigger to final record. Look for deterministic controls around probabilistic model steps. The system should validate structured outputs, enforce business rules outside the model, support idempotent writes, avoid duplicate actions, and expose retries and failures. Ask what changes when the model, prompt, connector, or policy is updated and whether customers can pin or test versions.

Interrogate data handling and privacy

Create a data-flow table for prompts, attachments, retrieved content, outputs, feedback, memory, and telemetry. For each, record purpose, data categories, controller or processor role, location, retention, encryption, access, deletion, and whether it is used to train or improve any model. Review the data processing agreement, subprocessor terms, cross-border transfer mechanism, assistance with rights requests, and breach notification.

Do not accept statements such as enterprise data is private without the underlying configuration and contract. Some services have different controls by plan, endpoint, region, or feature. Verify whether administrators can set retention, disable memory, prevent training, redact logs, and limit retrieval sources. If personal data is involved, determine whether the vendor supports the buyer's lawful basis, transparency, minimization, DPIA, and deletion obligations.

Evaluate security and agent permissions

Request current assurance reports or certifications where relevant, penetration-test scope and remediation process, vulnerability disclosure, secure development practices, encryption details, identity integration, multifactor authentication, role-based access control, tenant isolation, secrets management, backups, disaster recovery, and incident response. Certifications can support diligence but do not prove the proposed configuration is safe.

Inspect connector permissions at action and resource level. Require dedicated identities, least privilege, short-lived credentials where possible, separation between read and write tools, and approval for high-impact actions. Test prompt injection through user messages, documents, websites, and tool output. OWASP guidance is clear that external content is untrusted and that model output must not be the authorization layer.

Demand an evaluation method, not a headline accuracy

Ask what task the reported metric measures, on which data, at what time, with which model and settings, and how uncertainty was handled. A benchmark unrelated to your workflow provides little purchase evidence. The vendor should support customer-specific test sets, versioned evaluation runs, slice analysis, human review, and regression testing after material changes.

Build a pilot set containing common cases, high-value cases, edge cases, restricted requests, malicious instructions, ambiguous inputs, missing records, and system failures. Score task correctness, groundedness, policy compliance, serious error rate, latency, cost, and reviewer effort. Keep a list of failures with severity and cause. Do not average a harmful action into a generally good quality score.

Verify integration and operating readiness

Test the exact systems, authentication method, permissions, data volume, and network path intended for production. Confirm supported APIs, webhooks, error codes, rate limits, queues, retries, idempotency, sandbox availability, change management, and recovery from partial failure. Ask who owns connector maintenance when an upstream API changes.

Review logs, traces, metrics, exports, alert integrations, service status, support hours, escalation path, recovery objectives, maintenance windows, and customer notification. The vendor should make it possible to answer which identity initiated an action, which model and configuration ran, which tools were called, what policy checks occurred, and whether a person approved the result, without exposing unnecessary personal data.

Model price, contract, and exit

Price the expected and stressed workload. Include seats, tasks, tokens, storage, connectors, environments, support, implementation, overages, minimum commitments, and human review. Ask how failed calls, retries, long contexts, and multi-step agents are billed. Require notice and options for material price, model, subprocessor, security, or product changes.

The contract should allocate responsibilities for data, configuration, security, incidents, intellectual property, evaluation, support, and regulatory assistance. Define service levels and remedies that matter to the workflow. Specify export formats, transition support, deletion deadlines, deletion confirmation, retention exceptions, and treatment of backups and derived data. NIST supply-chain guidance treats due diligence as an acquisition and ongoing monitoring activity, not a one-time questionnaire.

Run a documented decision gate

Score demonstrated evidence as pass, partial, fail, or not verified. Keep mandatory controls separate from weighted preferences. Record assumptions, open risks, compensating controls, contract dependencies, and the person accepting residual risk. A conditional approval should list what must be completed before production access is granted.

Reassess the vendor at renewal and after material changes. Monitor incidents, service performance, model regressions, subprocessors, data practices, and permission expansion. The most credible supplier is not the one that claims zero risk; it is the one that can explain limitations, provide evidence, support bounded testing, and help the buyer operate the system transparently.

Sources and methodology

This article synthesizes the primary sources below as of the publication date. Forecasts and recommendations are directional scenarios, not guarantees; they should be tested against your workflow, data, risk tolerance, and current vendor documentation.

UK Government: Guidelines for AI procurement (accessed 2026-07-30)

National Institute of Standards and Technology: NIST SP 1326 Due Diligence Assessment Quick-Start Guide (accessed 2026-07-30)

National Institute of Standards and Technology: Software Security in Supply Chains Guidance (accessed 2026-07-30)

UK National Cyber Security Centre: Guidelines for secure AI system development (accessed 2026-07-30)

UK Information Commissioner's Office: AI and data protection risk toolkit (accessed 2026-07-30)