AI Vendor Selection: The 25-Point Evaluation Checklist

Use this in the evaluation

• Live demo on YOUR data, not theirs
• Quality benchmarked against your eval set, not the vendor's
• Latency at your concurrency, not single-call
• SOC 2 Type II report (current)
• HIPAA / GDPR / PCI commitments where relevant
• BAA available if you handle PHI
• Data residency and processing region commitments
• Sub-processor list and notification policy
• Encryption at rest and in transit, with key management options
• Bring-your-own-key support
• Retention and deletion policy for your data
• Training opt-out for your inputs and outputs
• Rate limits and burst capacity in writing
• SLA on uptime, latency, support response
• Incident notification SLA
• Audit logs you can export
• API stability and deprecation policy
• Pricing model — per-seat / per-token / per-call — and the failure mode at scale
• Volume commitments and unused-credit treatment
• Termination for convenience clause
• Data export format and timeline at exit
• Reference customers in your industry and size
• Roadmap fit for the next 12 months
• Onboarding and integration support (paid vs free)
• Financial stability — runway, revenue, last raise