Member-only story
Simplified Handwriting OCR with AnnoMemo
Earlier this year I wrote about using VLM models to do OCR on my terribly scribbly hand writing. Models like GPT-4o are actually quite good at interpreting my rubbish writing and converting it to markdown. However, my workflow for using these models was a bit fiddly.
I have just finished an early version of AnnoMemo, a telegram bot that can receive images of handwritten notes and respond with a transcription of them. AnnoMemo also integrates with the popular memos app. It will automatically upload the photo of your hand written notes alongside the transcription as a new note and include a link to that note in its telegram response.
AnnoMemo is a portmanteau of Annotation and Memorandum.
Motivation
I went through a phase of manually uploading photos to ChatGPT or my self hosted LLM portal and copying and pasting the resulting text into my notes app. There are a few friction points in this process including the need to take the photo with my phone’s camera app before opening Open Web UI since it currently doesn’t provide a way of launching the camera in-app. I also need to highlight and copy the response and paste it into my memos app of choice. Another fairly major annoyance is that if the OCR model does get some words wrong I have to go and find the page to make sure I remember what I…
