The model looked great in the lab, but at 2:00 in the morning, the same part looked different under the line lighting, and now we were also dealing with a PLC that had been on that cell for years.
Inspection modernization isn’t about installing a new camera on the line. You need an image that still looks usable on night shift, inference close enough to the station to keep up with the line, and some separation between the AI system and the controller so you’re not treating a PLC like a web API.
That’s really why we started with the decision path before we started replacing hardware: what does the system need to see, where does the decision need to happen, and who owns that decision when the model is not sure? Here are five lessons from modernizing a legacy inspection system with AI.
The Project Behind These Lessons
Rather than explaining these ideas in theory, I’ll walk you through a recent project to modernize quality inspection. Our customer was a multi-site manufacturer trying to introduce AI into an environment built long before AI existed. Its lines ran on equipment with 10 to 20 years of history. We already knew the model could find defects in a controlled environment. What we really needed to know was whether it could keep doing that on a production line that had never been designed around AI in the first place.
For the VP of Quality, the AI quality-control initiative was not a digital transformation effort. Every missed defect could travel further down the value chain, and every false reject could slow the line. And the challenge wasn’t just technical, because the people running the line still had to trust what we were putting in front of them. If the operators don’t trust it on the floor, it doesn’t really matter how good the demo looked.
Lesson 1: AI Cannot Fix Bad Camera Data
The first thing we had to get right was the image itself. If the lighting never gives the camera enough contrast to separate the defect from the background, the model doesn’t really have anything reliable to work with. If the defect is not physically separated from the background, you just end up running the wrong decision faster. Mature inspection programs usually start with lighting and optics, not with the neural network:
Depending on the part, that might mean coaxial light for a polished surface, dark-field light for a crack, a dome for complex geometry, or backlighting when you’re really measuring an edge.
At that point, the obvious question was whether we should just replace the cameras. And sure, new hardware might buy you a few more points of accuracy, but now you’re taking out equipment that already works and putting line uptime at risk. So instead of ripping everything out, we kept as much of the existing stack as we could and fixed the parts that were actually limiting the system.
In this case, my team first stabilized the imaging setup. Working side by side with the client’s manufacturing engineers, we redesigned the lighting around the materials and lines before retraining the model. We kept coming back to a pretty simple question: does this part look the same to the camera at 2:00 in the afternoon and 2:00 in the morning? Once the answer was close to yes, accuracy became the next constraint, followed by latency.
Lesson 2: The Cloud Can Train the Model, but the Line Needs the Edge
Once we had the image under control, the next problem was latency. On a high-speed line, the automated defect-detection decision must occur near the camera, not after the video stream travels to the cloud and back. The cloud was still useful for training and model management, but we didn’t want a pass/fail decision waiting on a network round trip.
My team splits the architecture into two roles. The cloud handled training and versioning. Inference moved to a Jetson-class box at the station. For this kind of cell, we would normally put the inference on a Jetson-class device at the station. Orin is enough for a lot of multi-camera inspection work, and you only need to move up to something heavier when you’re running a much larger vision model or combining several sensor streams on the same node. Either way, the basic architecture stays the same: the station makes the pass/fail decision locally, and the cloud handles the work that doesn’t have to happen inside the cycle time.
If a decision affects physical movement on the line, it should run close to the equipment. If the work involves training, it can live in the cloud. It’s easy to design this around what is convenient for IT, but once the line is moving fast enough, that network trip becomes part of the inspection problem.
Lesson 3: Treating a Legacy PLC Like an API is a Mistake
Running inference at the edge solved the latency problem, but it created the next question: how do you get the data you need from equipment that may have been on the line longer than anyone on the AI team?
Some controllers had been running since the late 1990s. Aggressive polling could overload the communication bus. This is one of the places where an IT mindset can get you in trouble. A legacy PLC is not a web service, and you can’t just keep polling it harder because the AI system wants more data.
So we put a gateway in between the AI system and the production controls. It could read from the older protocols at a rate the PLC could handle, buffer that data locally, and expose a cleaner interface upstream without letting the AI workload hammer the controller directly. That mattered because an overloaded PLC doesn’t just give you bad data. It can trip a watchdog and stop the line.
So, the important change was that the AI system no longer had to touch the production controller directly. Once a buffer separated AI from PLCs, operations could experiment without risking uptime.
Lesson 4: The Model Is Only as Current as the Quality Rules Behind It
Once optics and edge inference stabilized, the bottleneck moved to training data. At that point, we found that having more images was not necessarily the problem. The bigger question was whether the labels still matched what the quality team actually considered a defect today.
We used larger foundation models to do the first pass at annotation, so the inspector was correcting a mask rather than drawing every defect by hand. Tools in the SAM family can handle much of that pre-labeling now, and Grounding DINO can help seed the initial boxes. Once the labels were cleaned up, we distilled that knowledge into a much smaller model that could actually run at line speed. The important part is that the big model stays in the training workflow; it doesn’t need to sit in the takt-time loop on the production line.
At the same time, the team rebuilt the QA logic behind the inspection process. Some rules were outdated. Some existed only in operator practice. Some were buried in old tests that no longer reflected how the product actually behaved. Without this step, AI could have automated stale quality rules.
The team created one defect dictionary for the inspection scenarios that mattered. That pulled tribal knowledge out of people’s heads and into the system.
Lesson 5: The People on the Line Still Need the Final Say
We were never going to start by letting the model make every final quality decision on its own. In this project, the customer did not want AI to become a black box making final calls on critical parts. They needed a system that sped up inspection while keeping the final call with the people who know the product.
That is why the team built a human-in-the-loop workflow. The AI handled the first pass and routed uncertain or critical cases to senior engineers. We also used smaller vision-language models to add context around what the system was seeing, so instead of just saying “defect,” it could point to where the issue was. The expert still made the hard call; the AI just removed a lot of the routine screening work. Instead of spending time on routine checks, they could focus on ambiguous cases and system improvement.
That fit the way the team already worked because we weren’t taking the decision away from them; we were just giving them better information before they made it. For the plant, that operating model mattered as much as model accuracy.
What Actually Changed After Modernization
By the end of the project, model accuracy was only one of the changes. The imaging setup became stable enough for the model. Inference ran locally without depending on a cloud response. PLCs were protected through gateways and local caching. Annotation became faster and less expensive. The QA team finally had a current defect standard. Operators could see what the system flagged, why it flagged it, and who still owned the decision.
And that sequence became a pretty useful readiness test for the next line: get the optics stable, keep inference local, protect the PLC, agree on the defect dictionary, and make sure someone clearly owns the hold decision.
Before / After: Not Just a Tool, but a New Operating Model for Inspection
| Before Modernization | After Modernization |
| Inspection depended on unstable lighting and manual tuning. | Optical stations were stabilized for specific materials and defect types. |
| The AI pilot performed well in the lab but lost accuracy in the field. | Models were validated under production conditions and were run locally at the station. |
| Older PLCs could not be polled aggressively without risking the line. | Edge gateways isolated the PLCs and gave AI safe access to telemetry. |
| Annotation took months at $5–15 per image. | Foundation models handled the first pass, and inspectors corrected the masks. On this program, the accepted-label cost was brought below $1. |
| QA logic was scattered across tests, documentation, and team experience. | The team created a single source of truth for inspection scenarios. |
| Operators did not trust black-box decisions. | Small VLMs and human-in-the-loop workflows added transparency and control. |
What made the project work was that the architecture aligned with how the plant actually operated. We fixed the optics before blaming the model, kept the real-time decision-making at the station, protected the older PLCs behind a gateway, and treated annotation as something the quality team would maintain after launch. Most importantly, the people responsible for quality still owned the final decision.
Conclusion
What this project really showed us is that you cannot just bolt AI onto an older inspection system and expect the model to solve everything around it. The model is only one part of the system. You first have to agree on what counts as a defect and who owns the decision when something gets held. Then you need an imaging setup that produces the same usable signal across shifts, a way to run the decision close to the station, and enough separation between the AI system and the legacy controls that experimentation does not put production at risk. Once those pieces are working, the model becomes much easier to trust and much easier to move to the next line. And you do not need a greenfield plant to get there. You need one station where the process is stable enough to prove the approach, learn what breaks, and then build from there.
Ready to modernize your legacy inspection systems? Contact us today to discuss how a tailored AI solution can stabilize your production quality and reduce operational risk.
Frequently Asked Questions
-
Can the camera see the defect consistently?
If the answer is no, it is too early to start with the model. First, solve the physics of lighting, contrast, focal distance, synchronization, and image stability. AI needs a signal, not chaos.
The team did not look for one universal lighting setup. For printed circuit boards, they rebuilt coaxial stations because the priority was reducing stray reflections and seeing surface relief consistently. For microcracks, they tuned dark-field lighting, which makes surface breaks visible. For parts with complex geometry, they used dome lighting to reduce shadows and uneven illumination. They also optimized backlighting for measurement tasks. On high-speed sections, they added strobe synchronization so the camera captured a controlled moment instead of motion blur.
-
How will AI get data from legacy PLCs without putting the line at risk?
Directly polling legacy controllers can be risky. A safer architecture uses edge gateways, controlled polling rates, local caching, OPC-UA, and report-by-exception event filtering.
Instead of putting direct load on the PLCs, the team added an intermediate layer. A Modbus store with local caching allowed the gateway to poll controllers at a safe rate while the AI read prepared values from the gateway’s memory. Legacy protocols were translated into OPC UA, unnecessary traffic was filtered using report-by-exception and deadband logic, and raw signals were converted into usable objects with timestamps, variable names, and line context.
That layer removed a major concern for Operations: AI gained access to the data, while the older controllers stayed protected from IT-style traffic patterns.
-
Does the company have a current definition of a defect?
AI should not be trained on outdated tests, incomplete documentation, or informal rules. Before scaling, the company needs to rebuild a source of truth: what counts as a defect, who confirms it, how the standard changes, and which cases require human review.
The team reviewed old tests, documentation, and actual inspection behavior to separate current rules from historical noise. Some scenarios were obsolete, some were incomplete, and some existed only as informal knowledge held by experienced QA specialists. That logic could not simply be moved into AI. It first had to be aligned, cleaned up, and made repeatable.
The result was a single source of truth for the QA team: what exactly counts as a defect, which scenarios are critical, which rules are outdated, and which should be carried into automated inspection.
-
Where should people stay in the loop?
For critical scenarios, full autonomy may be premature. Human-in-the-loop review, explainability, and senior review do not slow down an AI program. They make it acceptable for real production, where trust matters as much as accuracy.
-
How knowledge from large models was moved into fast production models?
Foundation models were useful for labeling, but too heavy for real-time inference on the line. The team transferred their knowledge into smaller models optimized for the edge.
After preparing the datasets, the team used knowledge distillation to move the behavior of larger models into compact architectures suitable for the edge. TensorRT optimization reduced inference latency, while LoRA adaptation enabled tuning models for plant-specific defects without a retraining cycle.
SAM 2 was used for fast segmentation, Grounding DINO for locating objects and defect areas, and Florence-2 for multimodal image analysis. Importantly, these models did not become the production inspection engine. Their role was different: to speed up data preparation, reduce the cost of manual labeling, and give smaller models better training material.
That changed the economics of data preparation: annotation costs dropped from $5- $15 per image to less than $1 per image, and the team could update datasets more quickly to include new defect types.
This preserved accuracy while delivering the speed needed on a factory floor: decisions were made locally, with millisecond-level latency and no dependency on a cloud round trip.
-
How small VLMs gave the system a way to explain itself?
At this level, the team tested vision-language models, including SmolVLM, Llama 3.2 Vision, and InternVL 3.5. Their value was not only classification but also the ability to connect a visual signal to an explanation. An engineer saw more than a “defect” label. They saw context: where the anomaly appeared, which defect family it was closest to, and why the system recommended sending it for review.
That helped reduce the black-box feeling. Engineers received not just an alert but context they could verify, discuss, and use for future model improvement.
