Safety Surveillance

Area intrusion detection systems are crucial in industrial safety, preventing unauthorized access to hazardous areas. This technology is used to enhance safety measures around machinery, such as rotating or electrically energized equipment, which can be dangerous if accessed during operation.

Objective: Develop a system using computer vision to detect people entering restricted areas near dangerous machinery. Record the entry and exit times of each person into a database. Alert system operators when a person enters the restricted area during machinery operation.

Person Detection

Process a real-time video stream to detect any unauthorized access. Store relevant information such as the timestamp of entry and exit in a database for later review.

Implement person detection using pre-trained models capable of recognizing human figures in various postures and conditions. Suggested tools include MediaPipe Pose or YOLO (You Only Look Once) Tracking frameworks that offer real-time detection capabilities suitable for monitoring areas with potentially hazardous equipment. Example code is available for Object Detection, Pose Tracking, and Hand Tracking.

Image Classification and YOLO Models

Image classification assigns a label to an entire image rather than detecting and localizing multiple objects. In safety surveillance, classification can be used to identify scene-level conditions such as safe operation, person present, PPE compliant, or unsafe state. This complements object detection, which identifies and tracks individual people or objects within the frame.

YOLO Classification Algorithms

Ultralytics YOLO supports both object detection and image classification workflows. YOLO classification models (with -cls suffix) are optimized for fast, real-time inference and can be trained on custom datasets organized by class folders. These models predict the dominant class in a frame and are useful when the camera view focuses on a single object or scene. In safety applications, classification models can be trained to recognize:

Safe vs unsafe operating conditions
Presence or absence of PPE (helmets, vests)
Equipment states (running, idle, maintenance mode)
Restricted zone clear vs occupied

A key distinction is that classification is one label per image and detection is multiple objects with bounding boxes. For COCO-style object recognition (e.g., person, car), detection models are required.

Other Popular Real-Time Classification Models

Several lightweight neural networks are commonly used for real-time classification:

MobileNetV3 – optimized for mobile and edge devices with low latency
ShuffleNet V2 – efficient architecture for fast inference on limited hardware
EfficientNet-Lite – scalable models balancing accuracy and speed
ResNet – reliable baseline with optimized deployment options

These models are often used when deploying on embedded systems, GPUs, or CPUs with strict performance constraints.

Example: YOLO Image Classification

from ultralytics import YOLO
import matplotlib.pyplot as plt

# Load a pretrained classification model
model = YOLO("yolo26n.pt")

import urllib.request

# download image
url = 'http://apmonitor.com/pds/uploads/Main/students_walking.jpg'
urllib.request.urlretrieve(url, 'students_walking.jpg')

# Run classification on an image
results = model("students_walking.jpg")

# Plot result as an annotated image
im_bgr = results[0].plot()
im_rgb = im_bgr[..., ::-1] # convert BGR to RGB for matplotlib

# Show figure
plt.figure(figsize=(10, 7))
plt.imshow(im_rgb)
plt.axis("off")
plt.title("YOLO Detected Classes")
plt.show()

[$[Get Code]]

Example: YOLO Object Detection

import cv2
from ultralytics import YOLO

# Load detection model trained on COCO
model = YOLO("yolo26n.pt")
cap = cv2.VideoCapture(0)

while True:
ret, frame = cap.read()
if not ret:
break

results = model(frame, verbose=False)
annotated = results[0].plot()

cv2.imshow("YOLO COCO Detection", annotated)

if cv2.waitKey(1) & 0xFF == ord("q"):
break

cap.release()
cv2.destroyAllWindows()

[$[Get Code]]

Safety Monitoring

The system should monitor the operational status of the machinery, using input from control systems to determine when the equipment is active. During these times, the area around the machinery is considered restricted. The Temperature Control Lab can be used for real-time testing. Shut off the heaters if a person enters the restricted area around the TCLab when the heater is above a certain level, such as 50°C.

Database Integration

Implement a database to record entries and exits from the restricted area. This includes timestamps and possibly identification details if individual tracking is enabled.

Example Solution

Below is a sample with 50 seconds of video detection from a webcam. Database entries are added each time a unique ID is created for a person when they enter or leave the video frame.

  row  id           entry_time            exit_time
    0   1  2024-04-19 10:32:43  2024-04-19 10:32:57
    1   2  2024-04-19 10:33:02  2024-04-19 10:33:08
    2   5  2024-04-19 10:33:12  2024-04-19 10:33:12
    3   6  2024-04-19 10:33:12  2024-04-19 10:33:13
    4   8  2024-04-19 10:33:13                 None

The video frame is given a red tint when an intrusion is detected and a unique ID is assigned when the person enters the video frame. The unique ID is assigned to the person until they leave the frame and the exit time is recorded. The camera could be positioned to monitor only the restricted area or a subset of the frame could be selected as the restricted area.

References

MediaPipe, an open-source library for building multimodal applied machine learning pipelines
Ultralytics YOLO (You Only Look Once), a state-of-the-art, real-time object detection system

Machine Learning for Engineers

Safety Surveillance