Still struggling with how I want to archive/save emails

This has been discussed here a number of times previously, including by me, and this is not the first time I’ve brought this up. Still, I am still struggling with this question and so I am soliciting thoughts and advice once again. I’m not sure why I am having such a hard time figuring this out…

I want to be able to essentially archive my emails, in the sense of having pretty much anything that I might need to find again someday available. This is not necessarily the same thing has having an email related to a specific task or project that I choose to store in a file in a folder of other documents related to this project, but rather a simple/quick/low effort way to archive everything before I delete it.

I don’t want to simply rely on the archive function in Apple Mail or something similar, because my goal is to create this email store outside of any current email provider I am using. The reason is that as things change and evolve, some email providers also change and I may no longer be using a particular email system a year from now. I also have multiple email addresses and providers including gmail, my own email server (hosted by an outside ISP), several accounts in outlook.com related to different businesses, etc, and I want the email to all be stored in one archive.

My top three contenders right now:

  1. Use a workflow of printing emails to PDF and saving attachments in separate files. I wrote a KM workflow that does this using Mail’s Save of PDF menu item, and also exports the entire raw email message and then parses out attachments to save in separate files (I don’t do this part in AppleScript because there are bugs in AppleScript’s ability to save mail attachments that Apple has not fixed in multiple OS versions). The files are named with a standard filename format that includes a SHA255 hash of the message to ensure uniqueness, and tagged as ‘email’, ‘has attachments’, ‘mail attachment’, etc, and Hazel can sort the emails based on which account they came from.

Advantages include local files that can be backed up, sync’d, etc, easy searching with Spotlight or HoudahSpot, easy viewing as PDFs or other documents. However, the fact that a dialog pops up for each save makes it harder and glitch-prone to save multiple messages, and so right now the KM macro works on only one selected email at a time, and of course UI scripting (even with KM’s ability to select menu items) is always glitch prone. There is no way to script having Mail write an email to a PDF.

  1. Archive to DevonThink: I keep going back and forth on whether I want to use DevonThink or not. I can import multiple emails at once and DT does a reasonable job of it, and displays and searches them reliable - and can index them in Spotlight as well so HoudahSpot will work if DT’s own search does not. On the other hand, DT is a very heavy-weight app that may be overkill, and I keep going back and forth on just keeping things in a plain file system vs an app. In addition, I have written rules in DT to have imported emails automatically sorted to the correct database (work related, personal, etc) but have found DT’s rule execution unreliable, although I have not been able to troubleshoot why.

  2. Synology mail server: I created a mail server setup on my Synology, which allows me to just drag messages in AppleMail to this account and they get stored there. This is very quick and easy, but: sorting emails into individual folders once they get there is not very good; I created Mail rules but they don’t work unless specifically executed, which is problematic, searching is now only available in Mail.app, and I need to backup the Synology data since it is not subject to the standard BackBlaze and cloning on my Macs.

  3. I looked at EagleFiler, but it’s interface just didn’t work out well for me, plus if you want to actually work with any of the emails you pretty much need to reimport them into Mail.

I would be very interested to hear what other people have done or could suggest.

1 Like

If you had not already evaluated EagleFiler, it would have been my first recommendation. Being able to quickly find an email in EF, then double-click and open it in Apple Mail to forward, reply, etc. is all I need. Thanks to Apple’s increased “security” in Catalina, HoudahSpot can no longer search Apple Mail, so you are limited to Spotlight for searching. And, IMO, PDFs are useful only as reference material

I think the simplest solution would be to keep your messages in an email client. You could create a local folder on your Mac in Apple Mail, then use a rule to copy all incoming mail to the local Archive folder.

Or you could use a separate email client as your archive. Before my company started using a dedicated email archiver I did this for one of our senior executives using the Thunderbird email client. He worked remotely 99% of the time, so I set up TB on one of my utility macs to archive all his messages to the local HD.

Thunderbird has the added advantage of storing messages in standard MBOX files which can be reimported into many/most email including Outlook, Apple Mail, etc.

Either Apple Mail or Thunderbird can store a copy of your messages locally and easily copy/move them back to your preferred email provider using standard IMAP.

Eagle filer sounds like something I’m looking for but when I went to the site it struck me as old. They’re still showing the old Mac OS icon.
Is the sw updated often? Anybody work with it that can share some experiences?
I searched for reviews but they’re all really old.
Thanks

I discovered this program called E-mail Archiver years ago. I use that and have been very happy with it. Once a month, I have it archive all my emails. It directs them to certain folders that I selected. It saves the emails in folders by year and month and it extracts all the attachments. I used the E-mail Archiver Business version which had some features that I needed over the basic version. I’ve been very happy with it.

3 Likes

I have been using EagleFiler since at least 2007. And, coincidentally, it received an update last week. It’s my email archiver as well as the archive for everything I want to keep permanently. It uses standard files and folders but maintains an index which allows me to find things quickly.

My iPad is my primary device and I recently switched from keeping a copy of my more recent documents in Evernote to Apple Notes (yet again). I imported the Evernote .enex files into EagleFiler, where any duplicates were removed, then copied the newer items into Apple Notes. Now everything I want to keep goes first to EF then to Notes thanks to Hazel.


Update (2020-11-05): For those using/considering eaglefiler the latest update includes a new script to fully download Apple Mail messages.
1 Like

The biggest problem I’ve had with this is searching. When email is in your email application, you can search by the various fields in the database. Once you flatten that structure to a PDF or Evernote then you’ve lost your ability to easily find things.

I’d be interested to know what you decide on.

PDFs seems to be a cumbersome format to deal with. It’s essentially a one-way street that’s very inconvenient to convert back - or convert to anything, really. Setting up a mail server on a NAS seems very complicated.

I’m not sure if I’m missing something or if I’m thinking too simple here but…

Why not store them as simple .EML files?
You can literally just drag mails out of Apple Mail into a folder in Finder to do this - and I’m sure that can be scripted or automated as well.

  • No proprietary database, gives you one mail, one file. And the file format is pretty much a facto standard (see link above), and basically a text file at that
  • As such, the .eml message can be searched in spotlight
  • You can use Finder tags on the individual files
  • .eml messages can be previewed with Quick Look
  • Attachments are saved inline, can be opened up from the mail file’s Quick Look preview
  • .eml can be viewed in Apple Mail. Or other Mail applications, even cross-platform
  • and they can be re-imported as mails back into Apple Mail if need be

PS: This might be due to having MailMate installed in my particular setup, which comes with some sort of bundled Spotlight Importer as I just found out. Not sure how it looks on a Mac without spotlight.

That’s been my choice for over 20 years of emails that I had to clean out of my Apple Mail system as it was getting impossible to use. Rather than try to sort by work or personal for my archive I just selected to put everything in year groups. And I archive nearly all messages, orders for things bought, sent mail incoming etc. I routinely go back 5-10-15 or my all time high 30 years to get old messages that I need to review or use again so the power of DT is worth it for me.

1 Like

Thanks everyone. Advice much appreciated.

I certainly do like DevonThink and saving to DT is reasonably easy since they provide a mail plugin. However, as such as I want to like DT, I tend to find it more cumbersome than helpful compared to just keeping all of my “stuff” in various files accessed via Finder and searched via HoudahSpot. I find some cognitive dissonance with having some stuff in DT and some in files organized in Finder, and I just want to have one place to look for everything. (And yes, I considered indexing my whole file store in DT, but the problem with indexing is the problems with syncing across multiple computers, AND the fact that DT does not automatically reindex when new files are added AND creating a file or document in DT does not cause it to be stored in the indexed folder, so things get out of sync).

I don’t know if I want to keep using DT just for email storage…although I guess since I own a license I could.

I hear you, @AppliedMicro, regarding the portability and long term durability of various file formats. I would argue that PDF is not likely to disappear in my lifespan, although I agree some search ability is lost with PDFs vs EML files.

I have a KM macro that addresses this in the following way. When run, that KM macro saves the (single) selected message in Mail.app as an eml file. It using KM’s ability to control menu items to have mail export the message as a PDF. It runs a python script that parses the email message in the eml file and writes all attachments to individual files. The files are renamed using a prefix that contains the mail’s date and a SHA256 hash (which ensures that all of these related files have the same filename prefix, and that names are unique between different emails). The pdf is tagged as “email”, the EML file as “raw email”, the attachments as “mail attachment:” and all get a tag based on the email account the mail was received by (which Hazel can use to sort the emails to a folder system).

The problems here are many: It’s not actually that easy to create a KM macro that reliably handles multiple selected emails at once; there is the user interface popping up dialogs in the middle of everything to do the save to PDF; Scripting the user interface is often prone to breakage with new OS versions.

Maybe I should just use DT for email archiving after all…