Computer Vision as a Solution to Safety Challenges in Industrial Facilities

Maybe Artificial Intelligence could be the answer to many workplace safety questions.

According to computer science literature, Artificial Intelligence is defined as a technique that can mimic the behavior of the human brain.

In recent years, we have seen more and more industry 4.0 solutions whose technology involves Artificial Intelligence, voice analysis, and image detection as an integral part of the manufacturing activity. These may include improving productivity, detecting machine faults earlier, and controlling quality.

In this article, we will discuss how to apply computer vision technology to detect safety hazards in industrial facilities, such as factories and logistic centers, in order to prevent human-machine accidents in real-time.

When the Computer Sees Better Than We Do

Think of a safety officer seated in a control room filled with monitors that display the activities of the factory. In one screen we see the loading area and the intense activity of operational vehicles, in the second screen we see one of the factory's production halls, with many assembly lines and activities taking place. In a third screen, a warehouse of finished goods is displayed, with shelves arranged in racks that sometimes reach the ceiling. And, these are just three of a full grid of screens.

No person, no matter how skilled and focused he or she may be in his or her work, will be able to view all screens simultaneously, identify near-accidents, and provide effective alerts in real-time which can lead to the prevention of accidents.

In contrast, computers with vision analysis algorithms are capable of simultaneously analyzing all screens 24 hours a day, seven days a week, and stopping dangerous activity before it becomes an accident.

Utilizing Smart Technology 

Computer vision is one of the oldest fields of artificial intelligence. A major goal of computer vision is to extract or analyze visual information from images. While it is an integral part of the solution, it is not sufficient to address the complex problem we face on its own.

In the complex world of industrial safety, finding objects in space is not enough. We have to identify the type of object (Object Detection), as well as how it interacts with other objects in the plant. It is vital that we continually monitor the trajectory of the object, including the speed at which it moves and its direction, as well as predict where it will arrive and at what time. 

The most important thing is whether this action may generate potential risk for it or for another object (anomaly detection).

From Control Rooms to Autonomous Safety Systems

Let's return to the virtual control room cameras that were described in the previous paragraph. As a starting point, we will look at a camera located above the charging area in the factory. Objects moving in the area are recognized by the algorithms and classified according to their characteristics, such as a person, a forklift, a truck, a cart, a box etc.

The goal here is to ensure the safety of the people in the charging area, preventing them from being struck by vehicles. For every human-type object, an evaluation is conducted regarding its proximity to vehicles, both static or in motion. Moreover, in order to avoid creating false alarms, it is essential to make sure that alerts are given only in real risk situations and not in cases where there is proximity but at a safe distance.

We will now proceed to the production hall. The system checks whether workers entering the hall are wearing the protective equipment as specified by the safety officer for the working environment. Afterwards, the system analyzes the activities of each worker in front of the machines. The system has been taught the permitted and prohibited modes of operation of the machines, and it alerts when abnormal or dangerous behavior is detected.

An often-asked question is: "How does the algorithm determine what is right and what is wrong, what is dangerous and what is not?". The answer is that the algorithm is constantly learning. Objects, movement, and events captured by the system are fed and analyzed by the system to sharpen the distinction between safe and normal activity and life-threatening activity. 

A continuous learning process is a key element of machine learning algorithms and is responsible for preventing false positive alerts and ‘yelling’ only when an accident is imminent.

Deep Dive into Vision Analysis

An example that illustrates the issue of anomaly detection well is the distinction between the proper and dangerous use of a utility knife. On the above site, there are six machines, which are all located next to one another. The shelves adjacent to each of them contain the equipment necessary to operate the machines and perform production activities. The shelf contained screwdrivers, pliers, utility knives, and, yes, replacement blades. Things are going well so far.

The dangerous situation began when one of the workers realized his knife blade was no longer sharp enough and when he tried to replace it, there were no new blades available. So what did he do? In order to obtain a new blade, he approached the position of another worker. This is also fine. This is also acceptable. The problem was that he failed to remove the knife from his hand as he moved from machine to machine, and the blade, although it may not have been sharp enough to cut the raw materials, was sharp enough to stab the neighbor's arm and injure him.

Computer vision analysis has reached the point where it can identify not only a utility knife-type object, but also the exact circumstance under which it becomes dangerous to an employee. Hence, alerting in the precise moment critical for preventing injuries.

Turn off "Creative Bypasses"

Lastly, we'll look at how the video analysis provided by one camera, positioned in a strategic position and overlooking the whole operation area, secures the whole place. 

The example will be made more challenging by examining a state-of-the-art production hall, which contains multiple machines and robotic arms. These machines are essential for streamlining production since automated processes speed up production and shorten work processes, and are an integral part of the Industry 4.0 revolution, which is why almost all of these machines feature built in safety mechanisms.

This may seem to be the safest place in the factory, however, according to factory safety managers interviewed, accidents continue to occur despite the advanced safety features built into the machines. This is because it is the workers themselves who pose the greatest risk to their safety.

The workers keep finding ways to circumvent safety measures, placing themselves at constant risk of being injured and even killed.

Motivation is clear among the employees, they strive to achieve maximum output in order to meet the objectives set for them. While covering or removing the camera or motion sensor mounted on the machine, the worker does not stop to consider that he is putting himself or his fellow workers at risk.

The same algorithm of image analysis and computerized learning is employed here as well. During the first stage, the system trains itself to recognize appropriate working configurations, which utilize the built-in safety mechanisms. After several days and sometimes even several hours of training, the system begins to alert when anomalies arise and safety violations are committed. However, unlike the safety measures that are within the reach of the workers, the system cameras are not accessible to them and it is impossible for them to be sabotaged in order to disrupt their operations without documenting the damage.

Dynamic Safety

Workers in industrial plants and logistics warehouses are exposed to high levels of risk. Utilizing the technologies we looked at creates a comprehensive safety envelope for the whole facility, on all the different processes that go on daily. 

As a result of machine learning software, the system does not stop learning. The system detects obstacles, physical objects, and trends in dangerous activity among employees and assists managers in planning and controlling factory safety. 

Alert & Save Lives

If the risk detected reaches a near accident level, an alert must go off to stop the event from becoming a real disaster.

Several methods can be employed in order to accomplish this goal.

One of the most efficient ways of alerting workers is to use a flashing lamp integrated into a siren at a central location in the secured area as well as an audible alert. 

A major advantage of this method is its simplicity for implementation, as well as its immediate exposure of the dangerous behavior to all managers within the entire activity space.

The second level of accuracy is the installation of the alert tool on the machine itself.

In vehicles, the alert tool can be a simple light and sound alarm, as well as an information screen similar to those installed in cars.

When it comes to a production machine or assembly line, the alerting methods can include lighting and alarms, shutting off the machine, or integrating with the machine to eliminate just the specific risk rather than stopping the line's production altogether.

The use of traffic lights is another interesting environmental alert. Traffic lights have been used in factories for quite a few years, but they operate on a simple periodic basis. A traffic light that is operated by a computerized image analysis system, on the other hand, will display a red light only when there is a potential collision. This will save a significant amount of time and increase productivity.

Written by:  Maoz Tamir, CTO at ArmourSense, for OH&S.