File Storage and Backup Storage

My file and backup storage workflow functions like this:

I have 2 External hard drives connected to my 2009 iMac. A 1TB External Drive for Time Machine “TM” backups and a 2TB External Drive for all my personal files. I am trying to, but failing to store only applications and the OS on my iMac’s internal 2TB SSD because the files external HD keeps failing. I have now officially replaced that drive 3 times and the TM drive twice.

Can somebody please help me come up with a better solution for a working/daily storage drive? As well as a better solution for a time machine backup? Although, I must admit I am not a TM fan because should you ever need to restore from one of its backups - it will fail you! Trust me I speak from experience.

In addition to Time Machine, I trust my file backup to Carbon Copy Cloner (run twice each week). I’m also a Backblaze customer. The time and effort I’ve put into collecting my digital “assets” make the cost of multiple backup options worth the price for me.

As far as working/daily, my needs are probably different than yours. I keep all my working files on the iMac with the exception of my Lightroom library. For that I have a file where I keep the most recent photos. Everything else is on a 1TB USB drive, and backed up multiple places.

Hope you find a great solution for your needs.

I was never a fan of Time Machine. For my personal storage (internal and external storage) I clone my drives to separate external drives, and I also use BackBlaze for background cloud backup.

I recommend either SuperDuper! or Carbon Copy Cloner as backup software - SuperDuper! is much easier to use, while CCC is a little more complex but has a few more features. (I prefer clones because in case of drive failure I’d rather just plug in a clone and get right back to work than have to go out to buy a new drive then perform a TM restore.)

For external storage I’ve had good experiences with Seagate drives. I partitioned this $149 8Tb Seagate external drive to as the location for my internal and external drive copies. I keep most of my apps and files on my internal 3Tb Fusion Drive, but my main files drive is the Seagate Backup Plus 4TB USB 3.0 Portable External Hard Drive, which can be found for under $100.

You don’t say how long a time period this was, but if you’ve been running this setup for 10 years since the iMac was new in 2009, four drive replacements is not that out of line. Hard drives are mechanical devices that can fail.

My concern with your backup strategy is that everything is in the same place. If your home/office were burgled, had a fire, etc. everything would be gone.
To your setup I would add a drive kept offsite (perhaps even in your car if you have one), and Backblaze for a cloud backup.

@JohnAtl, I’ve recently signed up with Backblaze but still unclear how it would work exactly. My most important files sit on an external drive that I work off of constantly. I need that drive to not crash…ever.

Check the Backblaze site for a clear explanation how they work. Backups of all files, minus system files and apps, get automatically, continuously backed up in the background. Backblaze backs up files on internal or physically connected external drives, but not network-attached storage (NAS devices).

https://help.backblaze.com/hc/en-us/sections/203997348-How-It-Works

Thank you for explaining it to me!

I’ve used BackBlaze for a couple of years and have had no problems. (That said, I’ve never had to restore using it, either.) The main downsides are that it doesn’t copy apps and macOS, and that it only saves file versions for 30 days. But unlimited is unlimited - if you hung 100Tb of data off a Mac mini it would back it all up for the same $60/year.

Since I also do clones I have backups of my apps, so that doesn’t really bother me. But if for some reason you want everything backed up, and want unlimited files revisions saved, the main alternative is Crashplan.com, which has a Pro Business plan which costs twice what BackBlaze charges.

FYI Backblaze gets the nod as best consumer online backup service both by Wirecutter and The Sweet Setup.

Yes, 3 in 10 years. I need a better method for my working drive.

A few thoughts.

Firstly, as others have pointed out, three failed hard drives in a 10 year span is neither unusual nor unexpected. The rule is not to wonder IF your hard drive will fail, but rather WHEN.

In my view, the solution is not to seek out a setup in which a drive will never fail, because that is just not feasible, but rather a solution in which you can recover quickly and completely from a drive failure, and if at all possible, anticipate a failure before it happens as well.

Since you would prefer to store your data files on an external drive (not unreasonable; I actually do the same), it is important to consider that your 2009 iMac has, I think USB 2.0 as its fastest interface and so your data transfer speeds are going to be much slower than you would achieve using the internal drive. Still, since that works for you I am not suggesting you change it, but rather use a setup that achieves the above goals.

What I might suggest for your involves a number of things you can do; these are not mutually exclusive and provide a number of recovery options.

  1. Consider getting an external drive that actually contains two separate drives in a RAID1 configuration. Thereby the two drives are duplicates of each other, and as such, if one fails, you are still fully operational on the other until you buy a replacement and copy everything over.

  2. A second external drive of the same size would be used for a regular clone of the working drive. Software already mentioned includes SuperDuper!, Carbon Copy Cloner, and also ChronoSync, listed in order of ease of use (easy to hard) and capability/flexibiliy (least to most). I actually do a DAILY clone, so that if a drive crashes, my clone is at most 24 hours out of date. I personally use Carbon Copy Cloner but also used both others and all would work fine. I have CCC set up to save all files being removed (eg files deleted from the working drive that therefore should no longer be on the clone), and so I can also retrieve accidentally deleted files as well, and I would encourage doing this, but then you will need the clone drive to be bigger than the main drive to ensure enough space for these files. I periodically purge out these archival folders, every few months.

  3. Time machine is not a bad idea, which would be a third external drive, because while it is filled with bugs and unreliability, it often does wind up working, is a real-time backup, and therefore is worth having in place. I have restored a file or three from a TM backup over the years, and it’s quick and easy to do so.

  4. Finally, I totally agree with the other folks who have recommended that you have an offsite backup. If you care about your data, this is a must. Many people use BackBlaze which has the advantages of simplicity and low and fixed cost for unlimited data (from a single computer). As already noted, BackBlaze will backup data from directly connected external drives, provided they are connected at least once every 30 days. You would exclude the TimeMachine drive and the clone from your backup as being unnecessary and a waste of your network bandwidth.

My problem with BackBlaze is a simple one. Any file deleted from your computer (internal or external drive) will be removed from your BackBlaze store after 30 days. I have had a circumstance in the past in which I presumably accidentally deleted a large amount of data and did not realize it for about 6 months. How that happened I have no idea. With BackBlaze I would have been out of luck in data recovery. Since I use Arq instead, however, I was able to restore all of my data.

Arq, the program I use, is a more complex setup and does not come with storage; rather you point it at the online storage service of your choice (I use BackBlaze B2) and it does its thing. I have it set up to keep everything so I can go back in time much more than 30 days if necessary. Because all of my computers backup to the same B2 account, all of my data is in the same place and since I pay based on what I use, there is no issue of having more than one computer backing up on the same account. The cost is higher but worth the cost for me. The maker of Arq has also launched another produced called ArqBackup which provides a more “BackBlaze-like” setup that might be a better fit for you. Here you play for storage in 1TB allocations, and if you have several TB to store it will be much more expensive than BackBlaze, but because you pay more for more storage, you can save files indefinitely after deletion if you prefer. I have not tried this product as it would not be economical for me as I back up 4 Macs and by lumping all in to one store, I am not paying one fee for each computer which doe snot fully fill the pre-specified storage limit. I would encourage you to adopt one of these offline backup solutions.

Finally, people always ask what brand of drive to use, and frankly any and every drive manufacturer will have failures. People often point to BackBlaze’s yearly summary of device failure rates to use in vendor selection, but every vendor has had models that are more or less reliable than others, so there is a lot of variability that I think makes it hard to prefer once vendor over another. Seagate and Western Digital are the “big boys’” in this market and I think both make fine products. Both have a range of drives with some specifically geared toward “consumer” low burden use, and others geared towards the high end server continuous operation market. Some like to use only one brand feeling they are “the best” and others like to use different brands for their working drive vs their backup to hedge bets against a bad model. I use WD Reds (server grade) in my Drobo working drive, and a mix of WD Reds and Seagates in my backup Drobo, and while I have had good luck with both brands, I believe the correct solution is again to ensure adequate backup and recovery protocols are in place rather than to try to find a drive solution that will never fail.

Lastly: whatever you set up, make sure it is a solution that happens automatically, eg set up your clones to happen automatically every day/week or whatever schedule you want. If you rely on manually triggering clones you will forget or get busy or whatever, and no strategy can help you if you fail to use it.

Hope this is helpful.

4 Likes

I use options 2 and 4 that @nlippman outlined for the external drives on my Mac Mini. Each external drive gets cloned to a second external drive every night. Everything gets backed up to Backblaze. They also get backed up to a separate third and fourth hard drive every two weeks as my “shelf backups”.

I do 2, 3, and 4, but instead of cloning daily, I just run a versioned backup of the user folder daily. This is easy to set up with Chronosync and, I assume, also with the other cloning utilities.

I have a full clone scheduled weekly, but I don’t always let it run. I’m especially leery of running a full clone immediately after a system upgrade, until I know there are no “issues” the upgrade created.

I do a backup to a disk image before doing an upgrade to a new major version of macOS.

Yes, I agree. With a huge amount of space on my backup Drobo, it’s easy to do an archival clone of the system drive before any major upgrade. I do that to a sparsebunle names with the source computer name and the date, so I have a number of archived restore images.

Just as an aside, to point out the truth of hard drive failures.

I just finished reconfiguring my desk, which involved moving my external drives (a Drobo 5D3 which is my main data, and a Drobo 8D which holds all the clones/backups) to a new UPS. In my cleverness, I managed to not plug the 5D3 plug completely into the receptacle on the UPS, and after the restart, the Drobo was flashing at me to tell me it was rebuilding data. Looking at the flashing lights I saw that one of the WD Red drives was not being seen, so I quite effectively seemed to have trashed a drive in this process.

Fortunately, it’s a Drobo, so it is still in the process of rebuilding its data store, but there should be no data loss. There are clones on the 8D and backups in the cloud, so I don’t need to worry.

And, I happened to have an unused 3TB WD Red on the shelf, so I just dropped in the replacement drive to replace the failed drive.

This is my first failed WD Red, but I’ve had other drives fail in the past. Always have copies at home and offsite for any data you care about!

I just want to put in a word for redundant backups. More than once I discovered my drive had gone bad when I attempted to back it up. The problem is that inevitably means the backup will be ruined as well. So I maintain two clone backups and alternate between them.

And another thing, always verify that you can restore files from your backup. I once had a bad tape drive that seemed to be writing but apparently was not! And twice I’ve had computers at work that were supposed to be backed up nightly by IT but were not (this was at different employers!). So I recommend at work to “accidentally” lose a file and as IT to restore it. Better safe than sorry.

2 Likes