Computer vision is the field of AI concerned with enabling machines to interpret and act on visual information from images and video. It covers a family of tasks: deciding what an image shows, locating objects within it, outlining their exact shapes, reading text, tracking movement and describing scenes in language. Modern computer vision is built almost entirely on deep learning.
Key takeaways
- The main tasks are classification, detection, segmentation, recognition and generation.
- CNNs dominated from 2012; vision transformers joined from 2020.
- Models now work with text too, enabling captioning and visual question answering.
- Performance depends heavily on image quality and how well training data matches reality.
- Deployment raises privacy questions, especially for anything involving people.
Computer vision explained in simple terms
A camera captures light as numbers. A photograph to a computer is a grid of pixel values with no inherent meaning. Computer vision is the work of turning that grid into information a system can use: “this is a cracked weld”, “there are three people and a forklift”, “the label reads batch 4471”.
The main tasks
| Task | Question it answers | Output |
|---|---|---|
| Image classification | What is this image of? | A label |
| Object detection | What objects are here and where? | Boxes plus labels |
| Segmentation | Which pixels belong to what? | Pixel masks |
| OCR | What text does it contain? | Extracted text |
| Facial recognition | Whose face is this? | An identity match |
| Pose estimation | How is the body positioned? | Keypoints |
| Captioning | What is happening? | A sentence |
| Visual question answering | Answer a question about the image | Text answer |
| Tracking | Where does this object go over time? | Trajectories |
| Generative vision | Create or edit an image | New pixels |
How modern computer vision works
- Capture: images or video frames arrive from cameras, scans or files.
- Pre-process: resize, normalise, correct lighting, remove noise.
- Encode: a CNN or vision transformer converts pixels into features, learning edges, textures, parts and objects across layers.
- Predict: a task head produces labels, boxes, masks or text.
- Post-process: filter by confidence, remove overlapping detections, apply business rules.
A short history
Early systems used hand-designed features and struggled outside controlled conditions. The turning point was 2012, when a deep network sharply cut error rates on the ImageNet classification challenge. Object detection matured through the R-CNN family and YOLO, segmentation through U-Net and Mask R-CNN, and in 2020 vision transformers showed that attention-based models could match or beat CNNs given enough data. Today, multimodal models combine vision with language, letting users ask questions about images in plain English.
Where it is used
Manufacturing inspection, medical imaging support, retail shelf monitoring, document processing, agriculture, logistics, autonomous vehicles, security, sports analytics and accessibility tools all rely on computer vision. In business, the highest-value uses are usually unglamorous: reading documents, checking product quality and counting things accurately.
Limitations and risks
- Accuracy drops with poor lighting, motion blur, unusual angles or rare objects.
- Models can rely on background shortcuts rather than the object itself.
- Error rates can differ across demographic groups, which matters for anything involving people.
- Adversarial changes to images can fool models.
- Cameras capture more than intended, raising privacy and consent obligations.