Automating Part Counting with Fine-Tuned YOLO Models and Synthetic Data

Introduction

Accurately counting steel stamped parts on a conveyor belt presents unique challenges, especially when hundreds of unique parts need to be supported. Our solution involved fine-tuning YOLO models for each part type and maintaining a dynamic registry of models for deployment. Additionally, we tackled the complexity of counting pairs of parts with inconsistent occlusion—a scenario that required synthetic data generation. Here, we detail our approach, the challenges we faced, and the successful results of our system.

Problem Overview

The primary requirement was to count stamped parts as they moved along a conveyor belt just before being placed in a bin. The system needed to handle hundreds of unique parts, with the added complexity of reliably counting pairs of parts produced simultaneously. Fortunately, we always knew which part (or pair of parts) was being produced at any time.

For parts produced in pairs, inconsistent occlusion posed a significant challenge. Depending on their relative positions on the conveyor, parts could partially overlap in ways that varied run to run. This made accurate counting difficult without introducing additional processing steps.

The Solution: Fine-Tuned Models and Synthetic Data

Fine-Tuning YOLO Models

We decided to fine-tune a YOLO model for each unique part type. By maintaining a registry of models that could be dynamically deployed, the system could switch seamlessly between part types based on production data. This ensured that the models were specialized for each part, maximizing accuracy.

Tackling Occlusion with Synthetic Data

To address the inconsistent occlusion for pairs of parts, we developed a synthetic data generation pipeline. This allowed us to create training datasets that simulated real-world scenarios with varying degrees of occlusion.

Starting Point: We began with about 500 baseline images of each new part or part pair.
Synthetic Pipeline: Using these images, we generated a wide variety of synthetic scenarios, including different orientations, lighting conditions, and occlusion patterns.
Labeling: The pipeline automatically labeled the synthetic data, drastically reducing the time required for manual annotation.

This synthetic data approach ensured that the models were robust to occlusion and could accurately count parts in all configurations.

Add segmentation points

Segmentation model

Generate synthetic images

Streamlining Onboarding for Hundreds of Parts

We are set up to onboard new parts by scheduling time when the press is idle to run systematic trials. Parts would be sent down the conveyor in a variety of controlled orientations, providing consistent data for manual labeling. The manual labeling process is expected to be efficient and quick, taking only about 20 minutes per part. Once labeled, jobs for training new models could be automatically added to a queue.

By keeping all processes on-premises, we eliminated the need for internet connectivity, a key advantage for secure, industrial environments.

ERP Integration and Discrepancy Resolution

At the end of each production run, the total counts from the system were compared with the ERP records. Discrepancies were flagged and investigated, providing valuable insights for refining the synthetic data generation process.

This iterative feedback loop allowed us to continuously improve the accuracy of the system.

Results and Successes

After refining the synthetic data generation pipeline and training process, the system achieved consistently accurate counts, even for highly occluded pairs of parts. Key outcomes include:

Accuracy: Error rates dropped significantly, with near-perfect counts recorded over hundreds of production runs.
Efficiency: The synthetic data approach and automated training enable us to onboard hundreds of new parts in a single standard weekend of downtime.
Security: By keeping everything on-premises, we avoided potential cybersecurity risks associated with internet connectivity.

Conclusion

Our system for automating part counting with fine-tuned YOLO models and synthetic data has proven highly effective. By addressing challenges like inconsistent occlusion and simplifying the onboarding process, we developed a robust solution that integrates seamlessly with existing ERP systems. This approach not only meets the needs of today's production lines but also provides a scalable foundation for future growth.