Kapernikov enables Umicore to increase plant capacity
Kapernikov helped Umicore to make its precious metal recycling processes more efficient. This enable ...
Kapernikov develops promising proof of concept based on its KIPP platform
Deep learning might seem like a complex, abstract concept, but it has helped Kapernikov to solve very concrete challenges. The Kapernikov team recently developed a Proof of Concept (PoC), based on deep learning techniques, that enables to see whether people wear their safety helmets at work in designated areas.
Imagine you want to monitor a vast construction site or a large factory to see whether people wear their safety gear. Monitoring people 24/7 across a large area is almost impossible with human operators. However, cameras equipped with the right artificial intelligence are able to do this automatically.
Kapernikov was asked by one of its customers to investigate whether safety gear detection would work. The customer wanted to see if it was technically possible to detect whether people were wearing their safety helmets and generate an alert in case they weren’t. The proof of concept was recently finalized, with very promising results.
Methodology
People detection is becoming an increasingly established technology. However, it remains difficult to determine whether or not the detected person is wearing a helmet. This is why the Kapernikov team split up the project into two parts. The first part of the solution was detecting people in a camera image. The second part was classifying them into two different groups: those wearing a helmet and those who aren’t. This split up allowed the team to make use of already existing, robust people detectors and focus its efforts on the challenging part: the helmet wearing classification.
First, the classifying neural network was trained from scratch for the recognition of patterns. To this end, we used a large selection of photos of people with and without helmets.
For both parts of the solution – people detection and helmet classification – we could make use of open source neural network architectures. For the training of the classification model, we used a training and a validation set consisting of examples of people wearing helmets and a group not wearing helmets. We also expanded the data set, by horizontally flipping and rotating some of the images.
Promising results
The proof of concept delivered great results, as you can see in the video below. When someone is detected, a bounding box is generated around that person in the video image. The box color represents the probability with which a person can be determined as wearing or not wearing a safety helmet. The color scale is a continuum where green signifies ‘wearing a helmet’ and red ‘not wearing a helmet’. When the model is about 50% certain, the box is yellow.
Deep learning
“We are very satisfied with the results so far,” says Victor Pessers, who was part of the PoC team. “Although this is still a proof of concept that is not yet fully developed, it shows the possibilities of deep learning.”
According to Victor, deep learning holds many more possibilities for industry 4.0. Many human tasks that have a repetitive nature could easily be done automatically based on deep learning techniques: face recognition, voice recognition, localization and positioning, or anything where sensors or the internet of things are involved.
Victor is no stranger to deep learning. His background as a Doctor in Mathematics has helped him to look at deep learning from a different angle. “There are quite a few standard deep learning solutions around. But mathematics has helped me to look deeper into the working principles of deep learning and to see whether these deep learning networks are applied in the right way.”
Project challenges
The current PoC was limited in several ways. Most importantly, the data set we used was rather limited, because the collection of quality data from a wide variety of sources is a time-consuming process. More data would definitely generate better results.
We had also hoped that the selected neural network architecture and the collected training data would enable us to distinguish safety helmets from other headgear, such as baseball caps and knit caps. However, the video images generated at the customer site were not of a high enough resolution to provide good results.
Further narrowing down the region of interest (ROI) in the video image could also improve the network’s performance. With the current PoC, the outcome is sometimes negatively influenced by irrelevant visual details (e.g. other people standing close or a person not standing upright). A possible solution for this could be replacing the person detector by a pose estimator, such as OpenPose, a real-time multi-person system that is able to detect human body, hand, facial, and foot key points.
Finally, we could improve classification by tracking people in the video or even on the construction site, from camera to camera. Combining our helmet wearing probability score with additional information about a person’s position towards the camera (e.g. close by or in the background) could result in a more accurate classification.
Localizing people, objects and robots in industrial environments
Helmet detection is just one of the many possible outcomes of the Kapernikov Industrial Positioning Platform (KIPP). Find out how it can help your company reach its goals.
It seems like you're really digging this article.
Subscribe to our newsletter and stay up to date.
