Optical character recognition (OCR) is technology that converts images of text into machine-readable text. It turns scanned invoices, photographed forms, PDFs and screenshots into data that can be searched, extracted and processed. Modern OCR uses deep learning and extends well beyond character reading into document understanding: knowing which text is a total, a date or a table cell.
Key takeaways
- OCR reads text; document AI also understands structure and meaning.
- Modern systems handle printed text very accurately, handwriting less so.
- Tables, multi-column layouts and poor scans remain the hard cases.
- Multimodal LLMs can now read documents directly, often with better layout understanding.
- Accuracy is measured by character and word error rates, but field-level accuracy matters more in business.
OCR explained in simple terms
A scanned invoice is a picture. To a computer it is pixels, not “₹18,450”. OCR is the step that reads the picture and returns the characters, so the amount becomes a number your system can use.
How modern OCR works
- Pre-process: deskew, denoise, correct contrast and detect orientation.
- Detect text regions: find where text appears on the page.
- Recognise: a neural network reads each line or region, typically a CNN plus sequence model, or a transformer.
- Post-process: apply dictionaries, formats and validation rules, such as checking a date is plausible.
- Understand (optional): map text to fields using layout, keys and position.
Classical engines such as Tesseract work well on clean printed text. Deep-learning systems and cloud document AI services handle photographs, rotation, varied fonts and complex layouts much better.
What makes OCR hard
| Factor | Effect |
|---|---|
| Handwriting | Far higher error rates than print |
| Poor scans | Blur, shadows, skew and low contrast |
| Complex layouts | Multi-column, mixed tables and forms |
| Tables | Structure is as important as the text |
| Mixed scripts | Devanagari, Latin and numerals in one document |
| Stamps and signatures | Overlap text and confuse recognition |
| Photographs | Perspective distortion and uneven lighting |
A worked example
An accounts team processes 2,000 supplier invoices a month.
- OCR extracts all text with positions.
- A document model identifies fields: supplier name, invoice number, date, tax amount, total.
- Validation checks that line items sum to the total and the GST number matches the format.
- High-confidence extractions post automatically; low-confidence ones go to a reviewer.
- Corrections feed back into evaluation to track accuracy by field.
Note that overall character accuracy matters less than whether the total is correct. Field-level accuracy is the metric the business cares about.
OCR vs document AI vs multimodal models
| Approach | What it gives |
|---|---|
| Classical OCR | Raw text, sometimes with coordinates |
| Document AI services | Text plus fields, tables and structure |
| Multimodal LLMs | Reading plus reasoning: “does this invoice match this purchase order?” |
Multimodal models have made document reading far more flexible, though they can be slower and more expensive per page than dedicated OCR, and may hallucinate values rather than returning nothing when text is illegible.
Measuring accuracy
- Character error rate (CER) and word error rate (WER) for raw recognition.
- Field accuracy for extraction: the share of invoices where the total is exactly right.
- Straight-through rate: the share processed with no human touch.
Limitations
- Handwriting and degraded documents still cause errors.
- Tables and forms need structure-aware handling.
- Numbers and identifiers are the highest-cost errors, so validation is essential.
- Documents contain personal data, so privacy and retention rules apply.