A vision-language model (VLM) can help quality teams start a defect-detection pilot before they have a large labeled set of defects. It works from descriptions of acceptable and abnormal conditions, and some approaches improve with just a handful of examples. That cuts the task-specific training work needed to test an idea. It doesn't remove the need to validate results on your own line.
Rare defects create a cold-start problem. A conventional detector needs examples of each defect type before it can be trained, and those examples may not exist yet. A pre-trained VLM offers a different first step: check whether its existing visual knowledge already picks up the condition you care about, then add examples where they help. Research on zero-shot anomaly detection and few-shot industrial defect detection shows the approach is worth testing, though no study can tell you whether it will meet your plant's acceptance criteria.
Before you start: define the inspection task
Start with a specific decision. "Find defects" is too broad to evaluate. "Flag a visible chip along this edge for review" names a location, an observable condition and an action.
Agree with quality and production teams on:
- The unit of inspection: a whole part, a surface, a region or a single feature.
- Acceptable variation: marks, textures or dimensional differences that should not trigger a reject.
- Defect definitions: clear examples, including borderline cases that need human judgment.
- The cost of each error: a missed defect (false accept) versus a good part flagged (false reject).
- The current baseline: how parts are inspected today and how those decisions are recorded.
You also need images where the defect is actually visible. Lighting, motion blur, exposure and triggering all limit what any model can see, so check image capture before blaming the model. Our guide to why high-speed defect detection starts with a better image covers this in detail.
Step 1: Test zero-shot defect detection
A zero-shot test asks a model to handle your inspection task without training it on images from your line. The model itself is still pre-trained, and some research methods adapt it on other data first. AnomalyCLIP, for example, adapted a pre-trained VLM to focus on normal and abnormal regions and was evaluated across 17 anomaly-detection datasets. That's evidence transfer is worth testing, not a prediction for your parts.
Write defect descriptions an inspector could apply consistently, such as "continuous, unbroken edge" versus "visible chip interrupting the edge." Include acceptable look-alikes in your test set. If a normal texture keeps triggering alerts, rewording the prompt may not be enough; the images, inspection boundary or method may need to change.
Record the prompt and configuration for every test so you can tell whether a later improvement came from the model, a clearer defect definition or different images.
Step 2: Add a few examples where they help
If zero-shot results can't separate defects from acceptable variation, try a few-shot approach with a small set of labeled examples. Depending on the method, those examples guide the model at inference time or drive additional fine-tuning. Document exactly what you supplied and whether any training took place.
A 2026 study in Algorithms adapted YOLO-World, a vision-language detector, to the MVTec AD benchmark. Using an average of 11.8 defect instances per category, it outperformed conventional YOLO detectors in 12 of 15 categories, with its best result, 64.9% mAP@50, on the texture-heavy Tile category using only nine training instances. Those are benchmark figures, not production-line accuracy.
The useful question for your pilot is: how much does performance change as you add carefully selected examples? Prioritize defects that matter operationally and good parts that look deceptively similar. For more on how training data shapes inspection models, see training AI systems to augment human visual inspection.
Step 3: Validate AI visual inspection on your line
A promising demo is a reason to run a controlled pilot, not to replace your current inspection. Collect representative images across product variants, shifts and capture conditions. Have qualified reviewers settle disagreements between the model and the current method, and report false accepts and false rejects separately; a single summary score can hide the trade-off that matters most.
Test beyond familiar images. Benchmarks such as MVTec AD 2 deliberately include test conditions not seen in training, and your pilot should too.
In practice, a team might keep its existing inspection in place while the VLM flags candidate edge chips for human review. After reviewing misses and unnecessary flags across several part variants, the team decides whether to refine the defect definition, collect more examples, improve image capture or stop. Our guide on when to review AI inspection results covers how to set those review rules.
Pilot checklist
Before expanding a VLM defect-detection pilot, can your team answer yes to each of these?
- Is the target defect defined separately from acceptable variation?
- Are the defect and the relevant feature visible in the captured image?
- Are prompts, examples and any model adaptations documented?
- Are evaluation images kept separate from adaptation examples?
- Is there a recorded comparison with the current inspection method?
- Have reviewers examined both missed defects and false rejects?
- Have results been checked across part variants and capture conditions?
- Is there a clear rule for when a person reviews or overrides a model flag?
Limitations: less training work is not no work
A VLM can't detect a defect the camera doesn't capture clearly. It may mistake a harmless surface change for damage, struggle with unusual defect shapes, or behave differently when the product or lighting changes. A small example set makes a pilot easier to start but may not cover the full range of production conditions. Set acceptance criteria for the specific inspection decision before judging the results.
It also helps to separate image inspection from investigation after inspection. A text-only large language model (LLM) can't examine an image, but it can help organize findings and explore likely causes. Our article on agentic AI for root-cause investigation walks through that workflow.
Next step: run a narrow, measurable pilot
Pick one defect on one part family. Establish your current inspection baseline, confirm the images show the defect, and test a zero-shot approach against held-out cases. Add a small, documented set of examples only to fix a specific failure, then review false accepts and false rejects with the people who own the inspection decision.
The goal is to find out whether less task-specific training can produce a useful inspection candidate, and what evidence your line needs before relying on it.
Exploring AI visual inspection for your line? See how the Loopr platform supports inspection across aerospace, medical devices and other industries, or start a pilot with our team.
Frequently asked questions
What is zero-shot defect detection?
Zero-shot defect detection means asking a pre-trained model to identify defects on a new inspection task without training it on images from that task. The model relies on what it learned during pre-training and on text descriptions of normal and defective conditions.
How many images do you need for few-shot defect detection?
It depends on the method and the defect. In one 2026 benchmark study, a vision-language detector beat conventional detectors in most categories using an average of about 12 defect instances per category. On a real line, add examples gradually and measure whether each batch improves results on a separate test set.
Can a vision-language model replace manual visual inspection?
Not without validation. A VLM is best piloted alongside the current inspection method, with people reviewing flagged parts, until line-specific results show it meets your acceptance criteria for false accepts and false rejects.
What is the difference between a VLM and an LLM in quality inspection?
A vision-language model processes images and text together, so it can evaluate visual evidence. A text-only LLM can't see images, but it can help summarize inspection findings and support root-cause analysis.
