654: Data Storage in 2022

I use both. I think CCC backups are easier to recover from than Time Machine.

2 Likes

Excellent advice, thank you!

I presume that for the ā€œshelf driveā€ backups youā€™d recommend CCC?

+1 to disable Time Machine entirely and replace it with CCC or Chronosync

I love Macs in general but not Time Machine. The problem is that if your Time Machine backup is corrupted, you may have no warning until you try to restore it. And one error in the backup file may cause Time Machine to reject the entire backup.

CCC is infinitely more sophisticated in this regard.

2 Likes

Thatā€™s what I would recommend. That covers you if there is a bug or some silent failure (as @rkaplan said), in TimeMachine.
In my previous Backup palooza system, I used multiple drives, software, etc. to provide redundant coverage of these possibilities. It was overkill, I think.

(Just thinking out loud, so feel free to ignore the rest of this if uniterested :slight_smile: )

Arq
  • Arq to backup my iMac Proā€™s home folder and my large data drive (RAID-0 paired 2TiB SSDs) to:
    • An external drive every hour
    • My Synology NAS every three hours
    • My TrueNAS server once a week (itā€™s on a Dell T320, so basically a space heater, thus I only have it on for ~4hrs a week)
    • I have two shelf drives that I back up to every other month, one for odd months, one for even
    • Every two or three years, I retire a drive that Iā€™m backing up to and put it away for long-term storage. (All physically here at my house, so not ideal in that regard.)
Carbon Copy Cloner
  • CCC backs up my iMac Pro ( not the data drive) to an attached 1TiB SSD.
Backblaze
  • I pay for 1-year retention
  • Backblaze backs up everything on my iMac Pro and the data drive.
  • Backblaze also backs up my girlfriendā€™s MBP.
iPad / iPhone
  • iPad backs up to my iMac Pro when I can get it to work, may go back to iCloud backup
  • iPhone backs up to iCloud
Manjaro Linux
  • Backs up to my Synology once a day
Not backed up
  • My MBP syncs some folders between my iMac Pro, NAS, and Linux server. Nothing unique on it, so I donā€™t back it up.
  • My TrueNAS doesnā€™t back up anywhere. Everything there is the same as my Synology NAS, save for the operating system itself. It also runs ZFS and has 8x 3TiB drives, so most failures are covered. For those still reading, TrueNAS is running in a QEMU VM on Ubuntu and works like a charm.

I put a lot of this in place to help ensure there were no setbacks when working on my PhD. I analyzed large datasets of EEG and fMRI data, which is why I bought the iMac Pro, external SSD for large file storage, etc. Depending on my next gig, Iā€™ll revisit all this, perhaps paring it down some.

1 Like

As always, you are a wealth of information, thank you! I must say, I was living dangerously, but did not realize it, when I wrote my doctoral dissertation. I used Word and Endnote with no backups at all. I suppose ignorance was bliss, but only because nothing crashed! :grinning: :grimacing::sweat_smile:

1 Like

@glenthompson @rkaplan @JohnAtl Thanks for the advice. I just purchased CCC.

Much appreciated!

1 Like

Just a general reminder that iCloud deletes backups after 180 days.

Back in the day, we had a fire at my university. And you hear ā€œnot much did happenā€. Which is correct, the fire was quickly extinguished and the building was not really (structurally) affected.

What everybody misses in the story: a whole corridor (2nd basement) burned and several hundred student lockers. And countless half or 3/4 done theses, saved on 3,5" discs, along with folders full of copied research material, papers, notes. Many saved their work on TWO discs. Which fire doesnā€™t care about. The main perceived risk was ā€œdisc stops workingā€.

1 Like

That is so awful, I canā€™t imagine losing all of that work and having to start over! :flushed:

And the library was closed for months, because they had to clean all the soot. In those days, accessing papers from home was not possible (except catalog) and sources were on paper.

It was a huge mess.

I wasnā€™t affected, but the story (and knowing affected people) had an impact on me and certainly, my backup habits.

1 Like

Really useful episodeā€“thanks!

Iā€™m starting to think seriously about possible deep archival cloud storage, and wonder if anyone has thoughts/suggestions about the best options for this.

Iā€™m an academic, and as a result I have 15+ years of research projects from undergrad through today on external hard drives, totaling probably 10+ TB of files. A lot of this is work that Iā€™m unlikely to need often, but I might want to access it once or twice in the next decadeā€“and in this case, it could be very valuable to have available. My backup regime has these on a couple of different hard drives, and Iā€™m thinking now about how to get one offsite, but I also would really like to have a version in the cloud in case of a real emergency. Doing something like Backblaze is a little impractical because I donā€™t want to keep a hard drive with all this deep history plugged into my computer at all times.

Are there good options for (hopefully low-cost) archival cloud storage where data isnā€™t accessed often but itā€™s held with very low likelihood of any corruption? Maybe the right answer is the low-access-frequency options with Google Cloud or AWS, but I always find their interfaces awkward so it would be great if there were more user-friendly options. But maybe Iā€™m just in a bit of an odd position so those are the best options. Thanks a bunch for your thoughts!

Welcome to the forum. I use the Backblaze b2 service, which is similar to AWS. It was easy enough for me to figure out how to use. I am only storing about 350GB at present. Here is a link to a price comparison: Cloud Storage Pricing Comparison: Amazon S3 vs Azure vs B2

Here is the first hit on ā€œcold storageā€ in a search of the MPU forum. Many suggestions including my favorite name for a cold storage service, Amazon Glacier.

Well, good thing MPU talked about data storage as I hadnā€™t looked into mine and ā€œassumedā€ all was well in the world. Not thinking about the Macbook Pro I had replaced a couple months ago and when I heard the comments about the Drobo drives, I became a bit panicked because, yes, I have two drobo drivesā€¦looked into things and found that my new mac didnā€™t even recognize them. So after trying a few dozen things and still no drives, I hopped on OWC and got a new enclosure with their RAID software. Iā€™m hoping Iā€™ll be able to salvage the data on the one drobo (set up to mirror) and then use some of the drives in the other Drobo for just random back ups. I didnā€™t even realize Drobo wasnā€™t around, things had been working fine at least a couple months ago. Let you know how things go, if I can retrieve the data (shouldnā€™t be a problem, right?). :slight_smile:

3 Likes

I used to have backups of work/study material in my university days/early career. Simple ones, on an external drive.

Once I did not. During a thunderstorm, lightning fell onto my house killing a number of devices (not all of them though), and my (urgent) work was gone. Since then I make sure backups are always in place. BB is great for that, you donā€™t need to remember doing anything.

3 Likes

When I worked for a university IT department, we were housed in one of the two oldest buildings on campus, built in the 1920s (gasp! ). They replaced all the transformers. On the day they switched them over, someone mistakenly flipped a switch that set everything to 220 volts rather than 110. I was walking down the hallway after turning off an alarm system in a lab I took care of, when I heard a popping sound: all the lightbulbs in the ceiling lights were exploding, one by one as they came on.

Every server was fried. The UPS units were fried. All the networked servers and RAIDs were fried. High-quality power strips were melted to the floor. External powered speakers in classrooms were fried. And most of the desktops, in a three-floor-plus-basement building, were fried. It was more damaging than the earthquake, in terms of hardware loss.

Because we had off-site backups of the networked servers, and users could back up to the servers, no one lost more than 24 hours of data.

I am a devout believer in off-site backups, in addition to local.

3 Likes

Wow! Thatā€™s a horror story.
Lucky there werenā€™t fires!

maybe try just getting a fresh external ssd (and maybe do a clean install)?

1 Like

@afhussain Maybe try First Aid too.

1 Like

Iā€™ve always been really bad at backing things up - in as much as I havenā€™t really done it.

Iā€™m starting a new business in a couple of months though, so this episode came at the right time and I found it interesting, as itā€™s something Iā€™m aware that I really need to start doing.

Currently Iā€™m using a Synology NAS, primarily as media server for Plex, but Iā€™ve also recently been using it for backing up photos. Prior to Devonthink, it also housed key documents that I could access remotely. Those docs are now stored within DT (with the databases housed on the NAS).

My original plan was to use a Backblaze business plan to back up the NAS but, having read through this thread (and listened to the show), it feels like Iā€™d do much better to:

  • Move as much as I can onto my MBP. Itā€™s only 256 GB, which means I should be able to move my DT databases locally (although Iā€™d have to leave photos and films for Plex on the NAS)
  • Buy a couple of SSDs and use CCC to back up my machine on a monthly basis
  • Invest in Backblaze still - but to use their machine back up rather than NAS backup.

Does that sound good? My main concern is that my photos (from the NAS) arenā€™t being backed up but, Iā€™m assuming that they would be caught as part of the local back ups from my MBP and would obviously be in the Cloud with Apple (although Iā€™m ware that thatā€™s not a back up).

Thoughts? Thanks a lot in advance.