Introduction
AI visual inspection vendors number in the hundreds globally, ranging from enterprise platforms with thousands of deployments to startups with a handful of pilots. The evaluation process is difficult because technical capabilities are hard to assess from marketing materials and vendor demonstrations are optimized to avoid revealing weaknesses. These nine questions expose the gaps between what vendors claim and what they can actually deliver.
What are the first four questions to ask an AI visual inspection vendor?
Question one: what is your true positive rate and false positive rate on deployments in my industry, measured over a minimum of three months of production? Request actual data, not demo results. Vendors with genuine production deployments can provide this. Those who cannot have not achieved production-grade performance in comparable environments.
Question two: who owns the defect images and trained models created during deployment on my line? Some vendors retain ownership of your production data and use it to improve models sold to your competitors. Require a data ownership clause in writing before signing. Question three: how does your system handle model retraining when new defect types appear or when our product design changes? Retraining agility is the metric that determines long-term total cost of ownership. Question four: what happens to our inspection data if we end our contract with you? Data export and deletion procedures must be specified in the contract.
What are the next five questions to ask AI visual inspection vendors?
Question five: can you provide two reference contacts in manufacturing environments similar to ours that I can call directly? Written case studies are marketing; a 15-minute call with a quality manager at a reference site is evidence. Question six: what is your maximum response time for critical system faults and what does your SLA look like in writing? Verbal commitments are not binding. The SLA must be in the contract with defined remedies if response times are not met.
Question seven: what hardware are we locked into purchasing from you, and what is the replacement cost if that hardware becomes unavailable? Hardware lock-in is a long-term cost risk. Prefer vendors who support standard industrial camera hardware over proprietary sensors. Question eight: how does your system fail safely? A system that silently passes parts when a camera cable is damaged is worse than no inspection at all. Verify fault detection and safe-state behavior before deployment. Question nine: what is your data center location and what compliance certifications does your infrastructure hold for AI visual inspection vendors processing sensitive production imagery? Relevant standards include SOC 2 Type II for cloud-based systems and ISO 27001 for information security management.
How should you structure the vendor evaluation process for AI visual inspection?
Run evaluation in three phases. Phase one: document review. Request the nine items above in writing and score each vendor against your requirements. Eliminate any vendor who cannot provide written answers to the data ownership, SLA, and accuracy questions. Phase two: reference calls. Call at least two references for each shortlisted vendor. Ask the references the same questions you asked the vendor and compare the answers. Discrepancies between vendor claims and reference experiences identify overselling.
Phase three: paid proof of concept on your production line. Require each finalist vendor to run a four-week pilot on your actual production line using your actual defect types. Define acceptance criteria in advance: minimum true positive rate, maximum false positive rate, and maximum latency. Pay for the pilot but deduct it from the contract if the vendor wins. This aligns vendor incentives with your evaluation criteria rather than their demo environment.
What contract terms protect you after signing with an AI visual inspection vendor?
Three contract terms are non-negotiable. First, a performance warranty that requires the vendor to meet specified true positive and false positive rates for the first ninety days. If performance does not meet specification, the vendor must remediate at no cost. Second, a data ownership and deletion clause specifying that all production images and trained model weights belong to you and will be deleted from vendor systems within thirty days of contract termination. Third, a technology escrow clause for AI model weights, so you retain access to a version of the trained model even if the vendor ceases operations.
Frequently Asked Questions
How long should an AI visual inspection proof of concept run before making a purchase decision?
Four weeks minimum for a single product type. Eight to twelve weeks for multi-product lines or applications with seasonal defect pattern variation. Shorter pilots do not generate enough data to assess false positive rates reliably.
What integration support should AI visual inspection vendors provide during deployment?
Vendors should provide dedicated integration engineers during the commissioning phase at no extra charge. Integration support that is billed by the hour on top of the system cost is a common hidden cost that inflates total deployment expense.
Conclusion
AI visual inspection vendors vary widely in production capability, data practices, and support quality. The nine questions in this guide expose that variation before you commit. Require written answers to data ownership, SLA, and accuracy questions during evaluation. Run a paid proof of concept on your actual production line with defined acceptance criteria before signing a full deployment contract.
Ready to see AI visual inspection in action on your production line? Request a Jidoka Tech demo and get a defect detection assessment tailored to your product and line speed.