How to Validate a Vendor's AI Inspection Claims: The Questions Nobody Wants You to Ask

To assess an AI inspection claim, obtain the test method, sample size, confidence interval, test operator and product conditions. Report detection and false-reject rates separately at the same operating setting. Repeat on your product, packaging and production speed. Missing evidence means the claim cannot yet be evaluated for your line; it does not prove that every vendor or every published study lacks validation.

Start with the product and contaminant rather than the algorithm label. Compare food X-ray inspection systems against your test brief, and read What X-Ray Inspection CANNOT Detect: An Honest Limitations Guide before defining the sample set and acceptance limits.

Where the famous numbers actually come from

A figure such as 99.5% detection or 0.2% false rejects is incomplete without its denominator and test conditions. These are examples of claims a buyer may encounter, not validated performance figures for MIQI or the entire industry.

A technical report should identify the product matrix, contaminant material and size, clean and seeded sample counts, line speed and decision threshold. A number quoted without these details may be useful as a lead, but not as an acceptance criterion.

Independent studies and supplier tests answer different questions. Check what was tested and whether the result transfers to your application. This article does not claim an exhaustive literature review or that no independent validation exists.

If a supplier provides a study, record who funded and performed it, whether a held-out test set was used, and which settings were locked before testing. Supplier-generated evidence can be useful when its methods and limits are transparent.

Common misconceptions vs. what is actually true

Misconception 1: "AI" is a detection capability

It isn't. AI is a classification layer that sits on top of a physical measurement. In X-ray, the physics comes first: the machine images density differences, and whatever the detector cannot resolve, no model can recover. A neural network can help you decide whether an ambiguous grey blob is a bone fragment or a fold in the product. It cannot conjure contrast that the detector never captured. When a vendor answers a physics question with an AI answer, that is the moment to slow down.

Misconception 2: A higher detection rate is automatically better

Detection rate and false-reject rate move together. You can push detection up on any system by loosening the threshold — and you will pay in good product thrown in the reject bin. Any accuracy claim quoted without its matching false-reject figure, at the same threshold, on the same samples, is meaningless. Always demand the pair. If a vendor gives you one number, they have chosen the flattering half.

Misconception 3: The hard problem is the model

Zeegers and colleagues studied a CT-based method for generating X-ray training labels. Their reported result using fewer than ten representative objects belongs to that experimental workflow, not a universal training-data requirement for food inspection. Ask how your supplier establishes ground truth and keeps evaluation samples separate from training.

Misconception 4: A successful demo predicts line performance

A demo run on samples the vendor selected, at a speed the vendor chose, with contamination the vendor inserted in a known position, tells you the machine functions. It tells you close to nothing about your recall rate on your product at your throughput. Demos are a filter for gross incompetence, not a validation.

Misconception 5: More sensitivity beats the right technology

Detector technology also matters. METTLER TOLEDO’s X56 DXD+ product information describes a dual-energy X-ray system. This manufacturer source supports discussion of hardware options, not an independently verified detection percentage or a performance claim for MIQI equipment. Validate the actual contaminant in the intended product.

What the technology direction actually looks like

Two things are worth knowing so you can hold a technical conversation rather than a sales one.

Ishida’s IX-PD documentation describes direct-conversion dual-energy detection. Use the manufacturer’s stated configuration when comparing hardware; do not assign proprietary algorithms or performance improvements to a model without the corresponding model-specific evidence.

For a production line, inference latency and the time available to actuate the reject device are practical acceptance questions. Ask for the configured response time and test the complete detect-track-reject sequence at the proposed throughput, including communications or power faults.

The validation checklist: questions to put in writing

Print this. Send it as part of your RFQ. A vendor's willingness to answer is itself a data point — arguably the most useful one you will collect.

On the number itself

1. What exactly does your accuracy figure measure — detection rate, false-reject rate, or a blend? State both, at the same threshold. 2. What was the sample size? 3. What is the confidence interval, and at what confidence level? 4. Who ran the test: you, the customer, or an independent third party? Can we see the report? 5. What product, packaging, and line speed was it measured on? 6. On how many distinct production lines has this been reproduced?

On the AI specifically

7. What is the AI doing — image classification, thresholding, image reconstruction, or reject decisioning? Name the step. 8. Where did your training data come from, and how was the ground truth established: manual annotation, or a construction-based method? 9. How many images, and how many distinct contaminant types? 10. What happens when we change product recipe or packaging — does the model need retraining, who does it, how long does it take, and what does it cost? 11. Does the model run locally or in the cloud? If cloud, what is the line's behaviour when the link drops? 12. What is the inference latency at our stated line speed, and what is the throughput ceiling?

On accountability

13. Will you write the detection performance for our product into the contract as an acceptance criterion, tested on our samples at our site? 14. What is your remedy if the accepted performance is not met after installation? 15. Who owns the trained model and the images from our line? 16. What certifications does the machine actually hold, and can we see the certificate number — not the logo on a brochure?

Write the agreed performance, test samples, production conditions and remedies into the acceptance record. A useful comparison is based on what each supplier can demonstrate for the specified application.

How to run the test yourself

You do not need a laboratory to do better than a vendor demo. You need discipline.

Use a controlled sample set with your product, packaging, fill weight and relevant contaminant types at varied positions. Keep seeded samples identified, contained and segregated from saleable product under an approved procedure. Include clean products to measure false rejects and run at production speed; blind the operator where this can be done safely. Record detection and false-reject results with sample counts, conditions and a suitable confidence interval. Twenty trials are observations, not proof of a high-confidence universal detection rate. Repeat for relevant product changes.

Do this once and you will know more about the machine's real performance than any published figure will tell you.

Where MIQI stands, stated plainly

We build X-ray and metal detection equipment, so we have an interest here. We will state ours openly rather than pretend to neutrality.

Our MQ-XR series is a standard-configuration X-ray foreign-object inspection machine for bagged products, and MQ-XR-P is the high-precision variant. Our MQ-MD-C series metal detectors are available in standard and non-standard custom configurations. We are a source factory, and we support free sample testing — send us your actual product and your actual contaminants and we will run them and send you the results, including the failures. That offer exists precisely because of everything above: the only accuracy number worth anything to you is the one measured on your product.

Two things we will not do. We will not quote you an AI accuracy percentage we cannot substantiate with a method, a sample size, and a confidence interval. And we will not imply certifications we do not hold: MIQI does not currently hold CE or ISO 9001 certification. If a supplier's certification status matters to your procurement rules, you should ask every vendor for certificate numbers and verify them independently — including ours. A supplier who tells you the uncomfortable thing before you sign is more likely to tell you the uncomfortable thing after.

Talk to an engineer, not a salesperson. WhatsApp +86 152 1890 9599 or email 897874196@qq.com. Send your product spec, your line speed, and the contaminants you are actually worried about, and we will tell you honestly whether X-ray, metal detection, or a combination is the right answer for you — including when the answer is that none of them solves your problem.