Extracting text from images: OCR and AI vision, and when each one wins

Every few months someone declares OCR a solved problem. It isn't — it's a split problem. There are now two quite different technologies competing for the same job, and they fail in almost opposite ways. Choosing badly costs you either accuracy or money, and usually both.

The two approaches

Classical OCR — Tesseract, ABBYY, the engines inside most scanning apps — detects character shapes and matches them against learned glyph patterns. It is fast, cheap, deterministic, and it returns character positions, which matters more than people expect. It degrades gracefully on clean printed text at almost any volume.

Vision language models read the image the way a person does: they interpret it. They handle messy layouts, mixed languages, handwriting, and low-quality photographs far better than classical OCR. They also hallucinate. A model that cannot read a smudged digit will sometimes produce a plausible digit rather than admit failure — and it produces it with the same confidence as text it read correctly.

That last difference is the one that should drive your decision. Classical OCR gives you garbage that looks like garbage. A vision model can give you garbage that looks like an answer.

Match the tool to the failure you can tolerate

Clean printed documents at volume — contracts, reports, books, anything typeset. Classical OCR. It is an order of magnitude cheaper, and on clean text the accuracy gap has essentially closed.

Screenshots and UI captures. Either works. Screenshots are usually crisp and high-contrast, which is the best case for classical OCR. If you just need the text out of a screenshot, quick extraction tools like ScanRead.ai are built for exactly this and don't require setup.

Photographs of documents — a page shot at an angle under bad lighting. Vision models, clearly. Perspective distortion and uneven illumination are precisely what classical OCR handles worst.

Handwriting. Vision models, with human review. Handwriting recognition has improved enormously, but it is still the category where hallucination risk is highest, because the model is doing the most interpolation.

Tables and forms. This is the hard case, and the answer is "both." Classical OCR gives you the coordinates you need to reconstruct the grid; a vision model gives you the semantic reading of what each cell means. Tools that do this well run both and reconcile them.

Anything with numbers that matter — invoices, lab results, financial statements. Whichever engine you use, verify the digits. A transposed figure in an invoice is a materially different error from a misread word in a paragraph, and no current system is reliable enough to skip the check.

The step most people skip

The single largest accuracy gain in text extraction is not the engine. It's the input.

Deskew the page. Crop to the document. Increase contrast. Photograph in even light rather than direct sun, which blows out highlights and eats the characters underneath. A well-prepared image run through a mediocre engine beats a bad image run through a good one, consistently and by a wide margin.

If your source is a phone photo, most of your quality problem is upstream of the model.

Extraction is usually not the actual goal

Worth stating plainly, because it changes what "good" means: almost nobody wants a text file. They want a searchable archive, a translated document, a summary, or a spreadsheet.

That matters because it determines whether you need layout preservation. If the next step is translation, you need the structure — tables, columns, reading order — not just the characters, because a translated document that has lost its table is not a usable document. Scanned document translation is a different pipeline from text extraction for that reason: tools like Scanned.to and, for the document-translation end of it, 술술번역 keep the layout intact through the round trip rather than flattening the page to a string.

If the next step is search or summarization, layout barely matters and plain extracted text is fine.

Decide which of those you're doing before you pick a tool. It's the question that actually narrows the field.

A workable default

Start with classical OCR. Measure. Escalate to a vision model only for the documents that fail — and when you do escalate, budget for verification rather than assuming the better model removed the need for it.

For most document sets, that routing puts eighty to ninety percent of pages through the cheap path and reserves the expensive, higher-variance path for the pages that genuinely need it. Which is the right shape for a problem where the two available technologies are good at different things.