Reviewing the Best Paid and Open Source AI-Powered Handwriting OCR

Remarkably (for me), this essay started its life as a scribble in a notebook rather than something I typed into a markdown editor!

A few weeks ago, Tiago Forte made a video suggesting that people can use GPT-4 to capture their handwritten notes digitally. I’ve been looking for a “smart” OCR that can process my terribly scratchy, spidery handwriting for many years but none have quite cut the mustard. I thought, why not give it a go? To my absolute surprise, GPT did a reasonable job of parsing my scrawling and capturing text. I was seriously impressed.

Handwriting OCR has now gone from a fun toy and conversion piece to something I can actually, seriously use in my workflow. So then, Is GPT the best option I’ve got for this task or are there any FOSS tools and models that I could use instead?

An Experiment

In my experiment I use the following system prompt with each model:

Please transcribe these handwritten notes for me. Please use markdown formatting. Do not apply line wrapping to any paragraphs. Do try to capture headings and subheadings as well as any formatting such as bold or italic. Omit any text that has been scribbled out. Try your best to understand the writing and produce a first draft. If anything is unclear…