New Workflow for Storage, Search and Backup of all Documents

After changing how I store and organize all of my documents (per this post), I’ve reorganized how I organize and integrate my documents, notes, writing, research, DEVONthink, and my backups. This workflow provides significant redundancy without too much effort. And, it includes markdown formatted archives and backups of my Ulysses articles and Apple Notes.

Here are screenshots that I hope make this a little clearer.

Ulysses showing files exported to external archive folder in Finder
![Screen Shot 2022-08-07 at 2.36.43 PM|404x500]

(upload://7Kk9Z3h0MeMjqsaQsxSlivqoPNa.png)

My DEVONthink Databases

Screen Shot 2022-08-07 at 2.37.05 PM

Corresponding Finder Archive Folders

Finally, because I can convert nearly any text file in DT to markdown and I’m already exporting Ulysses and Apple notes as MD files to a corresponding archive folder, I can, if and as desired, access these files with Obsidian.

I think I’m now ready for the new school year! :slightly_smiling_face:

PS: I have set recurring reminders in Reminders for my exports and backups.

2 Likes

Your diagram implies everything goes into DT before being backed up, or is that in addition to other copies that are on your drive (maybe they’re indexed?).
Anywho, if so (i.e. everything in DT and only in DT), I wouldn’t want that single point of failure.

Also, does backblaze back up your iCloud drive? I don’t recall off the top of my head.

Everything is in Finder and downloaded and thus being backed up continuously by Backblaze and every other week to two external drives.

I also import from Finder all of these files to “archive” databases in DT as another form of “backup.” Once a month I export ALL DT files back to Finder (the main reason is that all of my research goes into DT initially, not Finder), which again, is continuously backed up. There is a good bit of redundancy. The only single point of failure would be my research files in DT but even then, the databases are being backed up by BB. I hope this makes sense.

But, by all means, if I have an error in this process–please tell me, I’d appreciate it!

1 Like

Good!
Trying to ride that line between “don’t assume ignorance” and “covering the basics” without offending. Humaning is hard sometimes.

2 Likes

You never have to worry about offending me. If I was offended by being corrected then the problem is mine–pride. And pride is a vice, not a virtue. :slightly_smiling_face:

4 Likes

I feel like I’ve seen you post something along these lines a couple times in the last month :rofl:

That being said, I love your backup system - reminds me that I should probably go through and make sure my data is being backed up adequately as well.

1 Like

Yes, because it has been evolving but I think “I have it”. This latest post was prompted by my reorganizing all of my documents by area/topic rather than by application. :slightly_smiling_face:

1 Like

This is the crucial part of your system. As long as you can live with losing everything since your last manual backup I see no problem with your plan.

I may be missing something and if so please correct me but even though I have downloaded all of my files they are still, I presume, being backed up via iCloud because I can still access them on my mobile devices. Am I understanding how this works correctly? If I am correct, then I don’t think I’m in danger of losing anything.

1 Like

The DT-Databases are a part of Finder, so why do you double the files in Finder?

I realize I’m creating duplications it the advantage is that rather than copying database files over into my external drives I am able to copy all of the actual files. Now, having said that, it could be that those files in DEVONthink databases are already assessable outside of the database in Finder and if so it’s something that I am not aware of.

I am open to a simpler way of doing this! :blush:

You can access any file, that is within a DT-Database by Finder, and the original Application.
You just has to do the search for the file, as the DT3-Database has an other structure, then the Groups showing up within DT3.

I have some big trouble in (mainly accidentally) duplicating a lot of my files over the last 2 decades by renewing my approaches how to stow them, by partial data loses, by new setups and installations from different Backups, from having different abandoned devices, that were saved before, and so on.
I currently have a very large Database within DT3, where I tried to copy all files from the different sources into, to remove over the next (I would assume 1-2 years!) all duplicates, and then insert those files back into my regular Database-Structur.
So, if you want to hear my advice, keep it simple!
Backup of course, but I would think twice in creating new duplicates of the files within finder and other places not origin to backups. I would keep this separate, one clean original place for my data (DT), and one or more Backups, but not two or more places to store my data parallel, specially if you can’t tell for sure the status of the different additional locations, and be on risk by that, to loose data in case DT really gets messed up, as you don’t know exactly at this point, which data where duplicated already, and which data had only one version within the DT-Database.

1 Like

This is great advice. I once tried keeping duplicates and ended up really frustrated as I had problems down the line finding the right version of files.

1 Like

iCloud backup doesn’t backup everything. AFAIK, for example, it doesn’t backup Apple Notes.

First, thank you for the advice. Upon reflection, I think you are right, I’m may e creating too many duplicates and in the process making this too complicated.

You made the statement “I would keep this separate, one clean original place for my data (DT), and one or more Backups…” Can you clarify? Regarding “and one or more Backups”, are you saying make one or more backups of the DT databases or in addition to the DT databases (which are being backed up by Backblaze and my manual backups to external drives) that I should also have one exported archive of my DT databases which is also backed up one or more times?

I like that workflow diagram @Bmosbacker. What tool did you use to make it?

iThoughtsX

2 Likes

The DT3 Database is still part of Finder, and your normal drive (If you haven’t moved the Database to an external location!).
So if you duplicate the Database back into the Finder, you will end up with several duplicates of the same (or versioned) Documents within the same place, and this will give you, sooner or later, some trouble.
I would import any document I use into Devonthink, and handle it from there.

I would have iCloud-Drive (which is not to be considered a “real” backup, but doing the job within its limitations!) doing its job on your Harddrive, and have in Addition a normal Backup like TimeMachine also that Backs the Harddrive.
Both “automatically” taking care also for your DT3-Database on the Harddrive.
If you want to have the extra portion of Back-Up Security, you can let Backblaze (or a similar service) doing an additional Backup in the Cloud, and having an App like CCC doing the same to one or more external Harddrives, you can switch on a regular base, and store them in a secure (of promise?) location.
If you want to do an extra amount of Backing Up, you could also use the rather new “Backup-Function” build into DT3 lately, but I would recommend to backup in this case to an additional location, and not onto the Harddrive itself.

With this, you have only one “real” version of your Documents, but several Back-ups on different locations, and with different conveniences in Restoring the Data, to be sure, that even in the worst case, you will end up with a reliable Backup.

4 Likes

@Ulli you convinced me, thanks! I’m doing most of the backups you recommend so it doesn’t seem necessary for me to export my DT databases back into the Finder each month. You are correct, doing so will likely cause me trouble eventually.

Thanks again for taking time to give good advice, much appreciated.

1 Like

This has been a fascinating discussion. Thank you.

How did you do the arrows in iThoughtsX?