EagleFiler to the Rescue

I’m posting this to celebrate an app many of you have already found helpful, and also to document my process in case anyone here runs into this issue.

I recently decommissioned an ancient Google account, which involved exporting almost 30GB of email using Google Takeout. I chose to keep it in a single file because I have fast internet and it seemed tidier. But I didn’t realize that all my power apps would hang up on importing such a large file. I haven’t been able to access the emails because I couldn’t find a program that could handle opening it. Thankfully EagleFiler was able to get the file open, and it didn’t even seem too taxing!

I downloaded the app, clicked Import File, selected the large .mbox file that I downloaded from Google, and waited a few minutes. It’s still “indexing” the nearly 400k messages, and it has been for a while, but I can search and access everything already.

Thanks, EagleFiler!

13 Likes

I have Eagfiler but never used it to the full extend. I think the developer Michael Tsai is on this forum as well. My experience is that it is great at finding files with relevant characters on the file name but I am struggling to get it to search for pdf file content by key words. This could be entirely due to my failure to configure Eaglefiler properly.

1 Like

There is a dropdown you can select in the EagleFiler search box for Filename or Anywhere.

thanks @karlnyhus , but I thought Eaglerfiler needs to OCR the content of the pdf files first. This is where I am having issue. It seems (I could be wrong) that it relies on PDFPen to add the text layer to the pdf file. This requires installing a script that I failed to get that working

EagleFiler is awesome. Blazingly fast, iCloud-aware while at the same time being purely filesystem based, thus compatible with other tools (I mix it with Notebooks.app) I only miss some sort of autocompletion for moving stuff around with the keyboard and better native Markdown support.

4 Likes

Out of curiosity, for folks that use EagleFiler to archive email, do you also keep the archived email in your provider account? (Gmail, iCloud, FastMail, etc…)

Whenever I consider archiving my email out of Mail.app, I get hung up on the fact that now I’ve got my mail in two places. It also kind of bugs me that the app is not fully Apple silicon compatible. Last I checked it still uses Rosetta.

Each year, I archive my e-mail OUT OF APPLE MAIL and than load it up in an EagleFiler library. Takes a load off of Apple Mail.

5 Likes

I haven’t seen a need for a separate OCR step. I just tried searching in EagleFiler and was able to find words within PDFs that I had simply Printed to PDF from the web.

Curious what you mean by this for EagleFiler. There is discussion once in a while on the EagleFiler Discourse about adding Markdown display and even Markdown editing into EagleFiler itself. I’ve always argued against complicating EagleFiler this way.

A simple double-click already will open a Markdown file in the Markdown editor of your choice.

Everyone has their favorite Markdown editor and flavor of Markdown. Choose EagleFiler for its plain text goodness and select the standalone Markdown editor that makes you happy.

This is how I got into needing to export my mail in the first place. Mail gets super slow to open over time, and I have to do this, which takes a considerable amount of time and requires that I set up all my preferences over again. I’m hoping to keep it lighter.

1 Like

Here is a statement from the developer, Michael Tsai (and I agree that performance in its current configuration is already exceptional).

That’s exactly what I was referring to. The thing is that RTF support is already there, so it feels like markdown is a second-class citizen. Even having a built-in Markdown preview would be nice.

Of course, this is a product rabbit hole: once you get native markdown, someone will ask for this or that flavor, dragging & dropping images, support for stylesheets, Mermaid… the list does not end :wink:

Edit: of course Markdown is a second-class citizen, because it is a second class citizen in macOS-land. RTF support is easy through the standard macOS frameworks.

And I’m okay with that!

Markdown is plain text and is intended to be readable (and is) with or without the visual sugar of a Markdown preview.

RTF is a word processing file format that you would never want to have to look in its raw form. :slightly_smiling_face:

1 Like

Yes. I have server side rules that filter my mail so only important messages reach my Inbox. But they also allow me to purge routine messages (delivery notices, e-bills, etc.) once they are no longer needed. I keep everything else.

Why not? Twenty years of email takes up less room than one movie and I can find any message almost instantly.

1 Like

I installed a Quicklook extension (GitHub - sbarex/QLMarkdown: macOS Quick Look extension for Markdown files.) which provides a prettier preview of Markdown files as you can tell Eaglefiler to use Quicklook rather than plain text. It means I have to double click to edit in my preferred app, but I’d do that anyway.

Eaglefiler is super fast in my experience, for indexing, capturing and searching. I used a competing product but regularly had database corruption errors - why I’ve no idea - and it explicitly didn’t work with iCloud disk saving options. I keep my EF library on iCloud - EF is able to handle and index iCloud properly, even if the save disk space option is enabled.

3 Likes

That’s my point. You have to click to preview Markdown after installing an extension, you have to open an external app to edit Markdown… that’s what I was referring to with markdown being a second class citizen.

I think we are both coming from DEVONthink. It’s also an exceptional tool, to be fair you can basically do what you want with it, and it finally cracked down the Markdown edition workflow pretty well imho, and has powerful smarts, but I would not say that they compete in the same field as Eagle Filer does, specially in terms of pricing and team size.

I am, too! I love EagleFiler!

1 Like

I don’t use EagleFiler, but I do archive my email (to a dedicated DevonThink database) and no, I don’t keep the email in my provider account.

Originally this was due to privacy concerns (ahhh gmail, a double-edged sword :joy:) and because I have several accounts and sometimes couldn’t remember where stuff was, but actually I’ve found that searching email via DT (in my case) is much faster and easier. The emails themselves are still email and can be opened by any email app if needed. I do this archiving weekly ish, manually! (I delete any rubbish, some things actually need importing elsewhere, etc.).

I personally recommend it, it makes finding that warranty email for the gizmo you bought 2 years ago so much easier!

1 Like

This is exactly what I want to do → save into EagleFiler then delete from Apple Mail.

My novice question: When I open an email that is saved in an EagleFiler library and it opens in Apple Mail does it save that email back into Apple Mail?

Glad to hear it—thanks for sharing your experience!

If you select Anywhere for the search scope it will search all of the PDF’s text content, too. If the PDF has no text content—because it’s an image/scan rather than a downloaded/printed PDF—currently EagleFiler will only search the PDF’s metadata. If your scanner software has OCR support or you have standalone OCR app, you can use that to add a text layer to the PDF, and EagleFiler will automatically recognize that. There’s also a script to help you do this in bulk for files that are already in EagleFiler. With Ventura adding support for Live Text, I’m working on adding built-in OCR support to EagleFiler. In the current public beta, this works for selecting text within a scanned PDF, and the plan is to extend this to indexing/searching, too.

Personally, I delete messages after importing them to EagleFiler. Otherwise, I’d be way over quota, and Apple Mail’s database and the Mac’s filesystem would be really bogged down. Some people do like to keep messages in Apple Mail after importing/archiving. You can use a separate mailbox, a colored flag, or a date-based smart mailbox to help keep track of which messages have already been imported.

You can use the TextFileTypesForQuickLook setting to make EagleFiler’s main window show Markdown files rendered using the QuickLook extension, no external app or extra click required. That said, I agree that it would be useful to expand the built-in support for Markdown.

By default, Mail is simply viewing EagleFiler’s copy of the message. However, you can use the Message ‣ Copy to command in Mail if you’d like to bring it back into Mail’s data store. Or you can bulk-import a whole mailbox from EagleFiler back into Mail, since EagleFiler stores it in mbox format, which Mail knows how to import.

6 Likes