Harnessing Handwritten Notes in Obsidian (An e-ink workflow)

[wug]

I’m both an Obsidian devotee and a great lover of handwriting. I believe I’m not alone in this — there’s been previous discussion of how best to integrate handwritten notes into a PKM, but no good solution. Goodnotes/Apple notes/reMarkable can convert handwriting to text (cumbersomely, on demand); but ideally one wants not just text transcription but also inclusion of any sketched images or handwritten equations.

After spending years doing this semi-manually (or not at all), I had one of those moments where the MPU stars aligned and I hit upon a Hazel+Shortcuts automation that does it nearly all. When I save a note from my E-ink tablet, this is what appears a few minutes later appended to my daily note in Obsidian:

Here’s a slightly rhapsodic write-up of the workflow: A Marriage between Handwritten Notes and Obsidian
In brief: Hazel retrieves notes from my E-ink tablet, runs them through a shortcut which makes an API call to Mathpix, then embeds the PDF and its transcription in Obsidian. The transcription quality continually impresses me, and Obsidian handles embedded PDFs so beautifully.

I’m using this with a Boox (android) e-ink tablet, but I imagine it could be easily adapted to the reMarkable/Kindle Scribe/just scanning pieces of paper. All you need is a way to get notes formatted as PDFs into a folder watchable by Hazel.

I’m curious how you’re all tackling this problem. MacSparky has often talked of the ‘Renaissance of Note-taking Apps’ – but while there’s an abundance of new zettelkasten-themed notes for typed text, there really doesn’t seem to be an equivalent Renaissance for the handwritten note. (The ‘All New Goodnotes’, for instance, remains stuck in that analog metaphor of files within folders; it seems all they’ve done is sprinkle gimmicky AI sauce on your handwriting.)

(Related - PKM with handwritten notes?)

[nationalinterest]

That really is superb. I prefer to handwrite notes in meetings, as it’s less of a barrier when listening to people and less formal than typing into a laptop, but I’ve been challenges as to how to feed them into Obsidian - at the moment I manually summarise them. The quality of the transcription now offered by AI is outstanding. And, of course, it could summarise the notes and the meeting documents too!

Good work!

I wonder how well local LLMs would work such as Llama 3 as a local alternative to Mathpix (which is new to me, but looks very useful.)

[DavidBeck]

Interesting workflow. Thanks for tagging my original note - I’ve still not landed on a good solution, partly through not having time to get as immersed in Obsidian as I want, particularly given complications with much of my PKM being related to a day job on PCs with enterprise IT rules.

[ehpehp]

Thanks for interesting post. I’m not able to get much coherent text from Mathpix reading my typical messy notetaking (on iPad). Notability is a little better but largely unusable. I do find value in sending PDFs of handwritten notes to Devonthink and searching based on note titles.