Backups (especially of large files)

Dear all,

I have a question which may be dumb, please bear with me.

I generally like keeping all of my important files on my Macbook Air - 1TB with High Sierra (main machine) to have with me at all times, except for TV shows and old photos which are stored on multiple external drives.

I backup my machine continuously with Backblaze, manually with Time Machine and occasionally do a Super Duper clone (I don’t like my external drives to be permanently attached, in the event of burglary, so I plug them in as needed).

My current predicament is that since COVID I’ve been collecting a lot of video material (mostly video recordings of conferences and classes, which I use to revise my notes) and they are too large to keep on my machine and quite resource-consuming for BB and TM backups too (especially since they both need to create local copies first, which eats up even more space).

I have therefore moved my large video recordings onto an external, permanently attached, drive (onto the Thunderbolt display which I attach the MBA to when at home). How would you suggest backing up these? I was thinking NAS, but I have to confess it’s not entirely clear to me how those work. Can I simply attach an external drive to my router with an ethernet cable and expect my MBA to see it? Or do I need something specific? My second thought was a drive attached to a Raspberry Pi in my partner’s apartment, would that work? (I would run the backup overnight since neither of us has much bandwidth). Does any of that make any sense and is it feasible? I have a 2TB Dropbox plan too (and I could select the folder not to sync), but I would prefer keeping the files on my own disks, if possible.

Thank you in advance for any inputs and apologies if I sound confused. It’s because I am.

https://www.backblaze.com :slight_smile:

1 Like

Thank you, but I’m already using Backblaze, I wanted to explore more local solutions or own networks.

So you already have them backing up to Backblaze, I’m assuming?

What’s the underlying reason for this backup? Do you want an off-site copy, or just another copy?

If you just want another copy you could get another external drive, plug it in on a weekend evening, and copy the new files overnight. There are a number of sync solutions that would allow you to effectively just copy the new files.

If these are huge files, I’d be sketchy on the effectiveness of running them over a network to a hand-rolled remote destination.

If your issue is more of “I have thirty billion external drives, and I wish I could put everything on one central, more reliable system” then a NAS with RAID might be a great option for you. A 4-bay NAS could be populated with huge drives, plugged into your Thunderbolt display, and you’d have some manner of redundancy.

But any NAS has the drawback that Backblaze won’t back it up. You might want to consider a more robust DAS solution.

QNAP has a TR-004, which is a DAS. That basically means that it plugs directly into your computer, and acts like a regular external hard drive - but it’s a 4-bay RAID unit. Backblaze sees it as a local drive, and is happy to back it up.

I have the rackmount version, and I’m very happy with it. You populate your own disks, pick a RAID level, and you have the best of all possible worlds. One giant bucket of local storage, some built-in redundancy (RAID provides some protection against drive failure), and the ability to have it all uploaded to Backblaze.

2 Likes

Thank you so much for this extensive explanation, it has clarified a lot.

The reason for not wanting to backup to Backblaze was more of a bandwidth + computer resources question. But I’ll think it through again, your input is very valuable.

Yeah, it’s definitely potentially a lot of bandwidth. But unless you have a bandwidth cap with your provider, if you just select the drive and let it run it’ll chip away at it until it’s finished. And if you really have something important that needs to be done, Backblaze lets you pause backups for awhile.

The nice thing is that Backblaze doesn’t have to keep sending the files over and over again. If you have a recording of a lecture today, yeah - it’s a hit on your bandwidth. But then it’s done. That particular file never has to be uploaded again. :slight_smile:

2 Likes

Well, there are workarounds :stuck_out_tongue_winking_eye: but in this case I probably wouldn’t recommend any of them!

1 Like

I did not realize that. Why not?

Because BackBlaze only backs up drives that are directly connected to the computer being backed up.

1 Like

Why do they make that limitation?

Presumably because NAS arrays can be huge, and they could effectively wind up backing up entire networks for $8 a month. :slight_smile:

OK I didn’t realize they price it for unlimited data.

That seems a bit odd considering how large SSDs are these days and how much installed storage is possible on a Mac Pro. And even more on a Windows server.

Seems to me 8Tb would be “effectively unlimited” backup storage for the overwhelming majority of users - and anyone who needs to backup more than that surely would understand why some additional fee is reasonable.

If you’re exposing a device to the internet, you’ll have to learn a lot, and maintain it to ensure safety.

If you go to your partner’s apartment, you could keep a backup drive there and attach it for a backup.

You could also keep a backup drive in your car, if you have one.

Then there are banks, you could keep one in a safe deposit box, rotate it out every once in a while with one you keep at home.

If your partner is physically distant (hopefully not emotionally), you could mail a drive back and forth at some interval. Safe deposit box is probably cheaper.

Thank you for this additional input.

Luckily, we live very close, so yes, bringing disks back and forth definitely an option. It wouldn’t be constantly up-to-date, but could be good enough (on top of my local backups).

As for the Pi, he’s the one managing it, I’m still very much a GUI, click-and-go type of person. This lectures are not sensitive data, but sure, if I were to transfer more personal data we would definitely need to make sure that we have all the safeguards.

How about CCC to a remote (off-site) always on Mac, previously encrypted with Arq or Cryptomator? Is that even an option?

I know this is an older topic, but hopefully you (and others) still get notified.

You can do that using tailscale, which is free.
It allows you to safely connect to computers from inside or outside your network.
Install Tailscale on both machines, then in CCC, you will use the Tailscale-assigned IP address for the remote machine as the Destination, then select a folder for the backup to be saved in.
I just tried this from my M1 MBP to my iMac Pro using CCC, and it’s working.
I’m not outside my network, but I’m using Tailscale for the connection.

That looks absolutely brilliant, thank you so much! I will try it ASAP.

1 Like

I have a follow-up question: this is not just simpler than forwarding a port in the remote router to my remote computer, it is also a lot safer, right?

What I like less is having to use a Google or Microsoft account, so I think I’ll create one on Github, I hope there won’t be any spying anywhere (data will be encrypted before leaving my machine anyway).

You’re welcome!

Right. It’s like a peer-to-peer, or mesh VPN. It doesn’t require opening ports to the internet, changing firewall rules, etc.

Here’s an overview, and a description of how it works.

I used my GitHub account too.
Tailscale purposely doesn’t handle accounts to increase security and to show they aren’t doing anything nefarious like being able to snoop data.

@JohnAtl I seem to have more questions.

I haven’t tried Tailscale yet. Do you have any idea of how it plays with NordVPN (which is installed on all my machines) and with Jump Desktop (which I use to connect to my “secondary” machines)? Will it break anything? I’ve casually googled that, so far no clear answer. Not sure if I should start a new topic with this question.