How to build an AI-powered Automatic Visual Inspection system from scratch Part 1

Part 1: Introduction to Visual Inspection

As Bob Ross once said, it’s the imperfections that make something beautiful. However, my colleagues who work in quality control might strongly disagree with him. One thing they fear the most during their jobs is deviations. If you work in a manufacturing company, you would wish all your finished products were as identical as copy-pasted.

To verify if the manufactured products have reached the desired level of perfection, a visual inspection process after production is inevitable. You might need to look for scratches on smartphone screens, foreign particles in pharmaceutical drug products, or missing components in semiconductors. Visual inspection guarantees the quality of your products and safeguards the safety and satisfaction of your customers. Without a proper inspection process, the company’s external failure costs will skyrocket. Here are some examples::

Typical External Failure Costs

For pharmaceutical companies this is even more critical, a mistake in the inspection process can lead to health and safety issues to their patients.

Evaulation of Current Status

Before we jump on the AI hype train and hire 50 data scientist, you should evaluate the current state mainly in three aspects:

1. current inspection process

2. current apprisal costs and external failure costs

3. current competencies.

Evaluation of current visual inspection process

There are three main categories of visual inspection: manual inspection, automatic visual inspection (AVI), and a middle-ground option called semi-automatic inspection.

1. Manual Inspection

Manual inspection is the most traditional method. Trained human inspectors use their eyes to identify known defects. They normally require proper lighting and a magnifier to identify particles and micro-components. Manual inspection is typically deployed when production volume is small or has a lot of variations because it is easy to ramp up and down. It is also used when expert human knowledge is required to identify the shortcomings of defective items.

However there are few downsides of manual inspection:

Mistake prone: Even though human inspectors need to go through rigours training and qulification process before they can work at the inspection station, it is still difficult for them to notice all defects (sometimes even major defects)within the time limits they are given, especailly when they are weary and tired by the end of a 4-hour shift.

Inconsistency: Human inspectors can treat products inconsistently due to subjective judgment. Variations in attention, fatigue, and personal bias can lead to different outcomes for similar defects, resulting in uneven quality control. An inspector with extremly sharp eyes might keep rejecting products while others find acceptable.

Injuries and phycological impact: Repetitive tasks and prolonged periods of intense concentration can lead to physical injuries such as repetitive strain injuries (RSIs) and eye strain. Additionally, the stress of maintaining high accuracy under pressure can cause psychological stress and burnout, affecting the overall well-being of inspectors.

2. Automatic Visual Inspection

To avoid the above-mentioned pitfalls, some companies decide to utilize Automatic Visual Inspection machine.

An Automatic Visual Inspection (AVI) machine uses a transportation system, cameras, and computer vision algorithms to classify whether a product is acceptable or defective. The classification results are sent to a programmable logic controller (PLC), which then directs the product to the acceptance or rejection area accordingly.

The most popular algorithms are something called rule-based model: rule-based models operate on predefined criteria and logic set by human experts. These models use specific rules to identify defects, such as size, shape, color, or other measurable attributes. For example, a rule-based system might be programmed to reject products with scratches longer than a certain length or color deviations beyond a specified range. These models are straightforward to implement and understand, but they can be limited in flexibility and adaptability to new or unforeseen defects.

Machine learning-based models have gained tremendous momentum in the past few years. These models are trained using large datasets of images, allowing the system to recognize patterns and anomalies that signify defects. Unlike rule-based models, machine learning models can adapt to a wide variety of defects and are particularly useful in complex inspection scenarios where defects are not easily defined by simple rules. Deep learning, a subset of machine learning, uses neural networks to provide even more advanced image recognition capabilities, making it highly effective for intricate and high-precision inspection tasks.

We will talk more about different kind of models in Part 4: Modelling

3. Semi-Automatic Visual Inspection

Semi-Automatic Visual Inspection typically involves human inspectors working alongside automated systems to enhance the inspection process’s accuracy and efficiency.

It reduces some of manual labours from the manual inspection, however majority of the pitfalls from manual inspection still exists in this process. It is possible to modify your Semi-Automatic machine and upgrade it to an Automatic machine by installing cameras and connect it to an edge machine.

Evaluation of current appraisal costs and external failure costs

When we initiate a new project, it is not only because it is fun and innovative but because it can save us money and time and bring in extra profits. So it is important to have a rough estimation of how much are current appraisal costs and external failure costs. By estimating the current state costs, we can better evaluate our project’s profitability.

The main components of appraisal costs are listed below

Typical Appraisal Costs

To implement an AI-powered inspection process, we will observe an increase of appraisal costs in the short-term and ideally a decrease of external failure costs. In the long run, after the solution is implemented ,as the AI system becomes more efficient and effective, we can expect the appraisal costs to also go down.

Another benefit of having Automatic Visual Inspection is that we can better understand the nature of our defective items. When we reject an item, the system will automatically document the defect categories in the batch report, reducing the costs on quality data gathering and reporting. We can utilize these data to improve quality training and decide which quality improvement projects the company should prioritize. Thus, we can reduce unnecessary spending on prevention costs.

Typical Prevention Costs

However, it is possible that implementing an AI-powered solution may not reduce costs even in the long run. In that case, it might be better to invest the money in other areas. So make sure to speak with your finance business patners to understand these costs before you begin your journey.

Evaluation of current competencies

Starting from scratch is always difficult, you can’t drive a successful project with only passion. I would suggest the initial team should compose with people that had at least 5 years experience in their respected fields Carefully evaluate if you have people with the below set of skills.

1. Inspection Specialists: Individuals who are familiar with all types of defects and know how to train and evaluate current manual inspection operators.
2. Data Scientists: The data scientists you are looking for are more like software engineers with expertise in computer vision and deep learning neural networks. They should know how to train machine learning models and deploy them on edge devices. Additionally, they should have a strong understanding of traditional rule-based computer vision models.
3. Robotics/Mechanical Engineers: People who can connect edge devices to the inspection machine, translate classification results into a language the inspection machine can understand through PLC, and design components using 3D modeling software such as AutoCAD. They should also be proficient with 3D printers. Sometimes, they also serve as the handyman for the project.
4. Vision Specialist: The photographer of the team. Someone experienced with cameras and lighting. They are knowledgeable about different types of cameras, filters, and lenses. They should also have experience setting up lighting conditions for photo sessions.