The AI Procurement Checklist

Most enterprises skip the structured procurement phase and jump straight to demos. This checklist forces rigor into the AI vendor evaluation process by breaking it into five stages: vendor vetting, RFP submission, demo preparation, POC setup, and contract review. Work through it sequentially before you sign an agreement.

Stage 1: Vendor Vetting (Pre-RFP)

• [ ] Define your work item. Specify the exact function (e.g., "password reset and billing inquiry tickets in English-speaking markets"). Avoid broad categories like "customer service" or "claims processing."
• [ ] Document your baseline. How many work items per month? Current cost per unit? Escalation rate? Quality metrics? This becomes your North Star for comparison.
• [ ] Build your vendor list. Identify 5–8 vendors in your category. At least one should be a tier-1 provider (OpenAI/Anthropic hosted solution), at least one should be an independent vendor.
• [ ] Initial vendor screen: Cost transparency. Call each vendor and ask point-blank: "What's your cost per resolved work item, and what assumptions does that include?" Rank vendors by willingness to be specific.
• [ ] Check for attribution API. Ask: "Can you expose cost per work item in your API responses?" Vendors with first-class attribution are rare. This is a primary differentiator.
• [ ] Verify compliance certifications. Do they have SOC2 Type II? HIPAA (if you're healthcare)? GDPR (if EU)? FedRAMP (if government)? Document each.
• [ ] Narrow to three finalists. Drop vendors who are opaque on cost or lack your required compliance certification.

Stage 2: RFP Submission

• [ ] Prepare your RFP. Don't use the vendor's template. Write your own with 40–50 questions (see "The AI RFP Template: 50 Questions Every Buyer Should Ask" for the full list).
• [ ] Cost modeling questions. Ask: What's the 95th percentile token usage per resolved work item? How does pricing scale above tier 1? What happens if I exceed volume? Is there a true-down clause if I use less?
• [ ] Attribution questions. Ask: What metadata is exposed in your API? Can you tag results by work-item ID? Do you offer batch cost reporting?
• [ ] Integration questions. Ask: How many API calls to my CRM/ticketing system are allowed? Are those charged separately? What's your latency SLA on tool calls?
• [ ] Lock-in questions. Ask: What's your model versioning policy? If you deprecate a model I depend on, what's my upgrade path? What data can I export?
• [ ] SLA and performance questions. Ask: What's your uptime SLA? What's your p99 latency? Do you have automatic rollback on model degradation?
• [ ] Compliance and data residency. Ask: Do you support data residency (US, EU, etc.)? How often are you audited? Can I audit your cost ledger?
• [ ] Set response deadline. Give vendors 2 weeks to respond. Vendors who miss the deadline are a red flag for execution quality.
• [ ] Score RFP responses. Use the vendor evaluation rubric from the pillar article to objectively compare responses. Multiply each dimension by its weight and rank vendors.

Stage 3: Demo Preparation

• [ ] Pre-demo briefing. Schedule a 60-minute call with each vendor. Spend the first 30 minutes briefing them on your work item, your baseline, and your success criteria.
• [ ] Request a custom demo. Don't accept a canned 30-minute demo. Ask for a 90-minute session with 60 minutes of Q&A.
• [ ] Prepare demo questions. Plan to ask: (1) Can you show me your cost attribution interface live? (2) Pull up a specific resolved work item and walk through the cost breakdown. (3) What's your 95th percentile cost per work item for similar customers?
• [ ] Invite the right people. Include your CFO, your operations leader, and a technical lead who understands your stack.
• [ ] Ask the vendor to bring a technical lead. You want answers, not sales slides. Make sure the person in the demo can speak to cost and integration.
• [ ] Request a sandbox environment. Ask the vendor to provision a test instance where you can run sample work items and see cost in real time.
• [ ] Record the demo. Get permission to record and share with your team.

Stage 4: POC Setup

• [ ] Define success metrics upfront. Don't start a POC with "see if it works." Be specific: "Cost per work item at $X ±$Y," "Escalation rate below Z%," "Quality score above W%."
• [ ] Set a POC duration. 8–12 weeks is standard. Most learning happens in weeks 1–4. Weeks 5–12 are confirmation.
• [ ] Agree on cost baseline with vendor. Before you deploy, write down the vendor's forecast: "We predict your cost per resolved ticket will be $0.55 at 5,000 tickets per month."
• [ ] Set up attribution infrastructure. Ensure the vendor tags every API call with the work-item ID. Ensure you can ingest cost data and join it with outcome data daily.
• [ ] Establish a weekly steering committee. Meet with the vendor and your internal team weekly to review cost, quality, and integration issues.
• [ ] Define go/no-go decision point. At week 6, measure actual cost-per-outcome vs. forecast. If within 15%, continue. If not, either renegotiate or terminate.
• [ ] Confirm escalation and quality tracking. Make sure you're measuring not just volume but also escalation rate, resolution quality, and customer satisfaction.
• [ ] Plan the scaling trigger. Define what volume or timeline triggers the transition from POC to production. (E.g., "After 2 weeks at 5,000 tickets/month with cost stable, we scale to 20,000 tickets/month.")

Stage 5: Contract Review

• [ ] Baseline cost-per-outcome is in the contract. Document your agreed cost target (e.g., "$0.55 per ticket ±$0.10") and the tolerance band.
• [ ] True-up and true-down clauses are defined. If volume exceeds forecast, you pay overages. If volume is lower, you get a credit. Both directions must be explicit.
• [ ] MFN (most-favored-nations) clause is included. If the vendor gives a lower rate to a peer, you get it.
• [ ] Model deprecation and upgrade path is specified. If Anthropic deprecates Claude 3.5 Sonnet, what's your upgrade path? Can you stay on the current model?
• [ ] Data ownership and export rights are clear. You own all data the agent processes. You can export in standard format (JSON, CSV, Parquet). No 30-day delay on exports.
• [ ] Termination rights are favorable. You can terminate with 30 days' notice without penalty if the vendor is in material breach (e.g., cost exceeds baseline by >20%).
• [ ] Audit rights are in the contract. You can audit the vendor's cost ledger annually. Vendor is obligated to provide logs supporting their invoiced cost.
• [ ] Change-of-control clause. What happens if the vendor is acquired or changes ownership? Do you have the right to terminate or renegotiate?
• [ ] Insurance and indemnification. Vendor carries E&O insurance. Vendor indemnifies you if they infringe third-party IP or violate data residency laws.
• [ ] Pricing lock period. How long does your quoted rate hold? (Typical: 12 months. After 12 months, the vendor can raise prices, but usually with 60 days' notice and a cap, e.g., +5% annually.)
• [ ] SLA penalties. If uptime falls below the SLA, what's the credit? (Typical: 10% credit for 99.0–99.5%, 25% credit for 98–99%, 50% credit for <98%.)
• [ ] Legal review. Have your legal team review the contract, especially data, indemnification, and liability clauses.
• [ ] Sign and document. Document the contract alongside your baseline metrics and success criteria. This is your reference for disputes later.

Before you close the vendor selection

Run this final check before you sign:

• [ ] Vendor has published SLAs and will honor them in writing.
• [ ] Cost attribution is real-time, not monthly, and tagged by work-item ID.
• [ ] Your team has tested integration with your stack and confirmed latency is acceptable.
• [ ] POC success metrics are written down and agreed by both parties.
• [ ] Contract has true-down, MFN, and audit clauses.
• [ ] Compliance certifications are in place and verified.
• [ ] Vendor has a technical support team assigned to you.

For a deeper framework on vendor evaluation, see "How to Buy AI: The Executive's Vendor Evaluation Guide." For specific RFP language, see "The AI RFP Template: 50 Questions Every Buyer Should Ask."