Object detection is the computer vision task of finding every object of interest in an image and marking where each one is with a bounding box, plus a class label and confidence score. Unlike classification, which labels the whole image, detection answers “what objects are present, and where?” It underpins autonomous vehicles, retail shelf monitoring, safety systems and industrial inspection.
Key takeaways
- Output is a list of boxes, each with a class and confidence.
- IoU measures how well a predicted box overlaps the true box.
- mAP (mean average precision) is the standard benchmark metric.
- One-stage detectors like YOLO favour speed; two-stage detectors favour accuracy.
- Non-maximum suppression removes duplicate boxes for the same object.
Object detection explained in simple terms
Given a photo of a warehouse, classification says “warehouse”. Detection draws a rectangle around each forklift, each pallet and each person, and labels them. It can find many objects of many types in one image, including overlapping ones.
How detectors work
- A backbone network, a CNN or transformer, extracts features from the image.
- The model proposes candidate regions or predicts boxes directly across a grid.
- For each candidate, it predicts a class and refines the box coordinates.
- Non-maximum suppression removes overlapping duplicate predictions.
- A confidence threshold filters weak detections.
Model families
| Family | Approach | Character |
|---|---|---|
| R-CNN, Fast/Faster R-CNN | Propose regions, then classify them | Accurate, slower |
| YOLO | Predict boxes directly in one pass | Fast, real-time capable |
| SSD | Single-shot multi-scale detection | Fast, lightweight |
| RetinaNet | One-stage with focal loss for imbalance | Good accuracy-speed balance |
| DETR | Transformer-based, set prediction | No hand-designed anchors or NMS |
| Open-vocabulary detectors | Detect classes described in text | Finds objects not in the training labels |
YOLO models, first introduced in 2015, remain the common choice for real-time applications; transformer-based detectors have grown since 2020.
Measuring detection quality
IoU (intersection over union) divides the overlapping area of the predicted and true boxes by their combined area. A prediction usually counts as correct at IoU ≥ 0.5.
Worked example: a predicted box overlaps the true box over an area of 600 px², and together they cover 1,000 px². IoU = 600 ÷ 1,000 = 0.6, which counts as a correct detection at the 0.5 threshold but not at 0.75.
mAP averages precision across recall levels and classes. “[email protected]” uses a single IoU threshold; “[email protected]:0.95” averages across thresholds and is stricter.
Practical considerations
- Annotation cost: every object in every training image needs a box, making labelling far more expensive than classification.
- Small objects: distant or tiny objects are much harder; higher resolution helps but costs compute.
- Crowding and occlusion: overlapping objects confuse both detection and suppression.
- Class imbalance: most image regions contain no object, which is why focal loss was developed.
- Edge deployment: smaller models and quantization allow detection on cameras and devices.
Common uses
Safety compliance checks such as helmet or vest detection, retail shelf and stock monitoring, vehicle and pedestrian detection, defect location in manufacturing, counting in agriculture and logistics, and sports analytics.
Limitations
- Boxes are coarse; use segmentation for exact shapes.
- Accuracy falls sharply in poor lighting, motion blur or unusual angles.
- Fixed class lists miss anything not trained for, unless using open-vocabulary models.
- Real-time performance requires trading accuracy for speed.