654: Data Storage in 2022

[mpubot]

Mac Power Users #654: Data Storage in 2022 - Relay FM

[Clarke_Ching]

That was a very useful session, thank you!

Question: could someone link to (the amazon.com page for) an external SSD drive that’d be good for connecting to an M1 Mac Mini and using it for time machine backups?

[afhussain]

@ismh Could you please let me know which internal SSD you used for the 2017 iMac? I have a 2017 iMac and had been running off an external Samsung T5 SSD but I was getting really slow boot times and other issues with hanging. I have now resorted back to using the internal 1TB fusion drive but like the idea of replacing it with a proper SSD.

I will watch some videos to see how daunting a task it is to remove the iMac screen before deciding.

[ismh]

I like the 2.5 inch Samsung Evo line

[JohnAtl]

I realize every piece of software and every service in these spaces can’t be covered, so here are a few I use:

• ArqBackup
  • very reliable (more so than Time Machine)
  • Supports many, many backup targets from Amazon S3 to Backblaze B2 to external drives.
  • Supports de-duplication, so you can have more backup history without a lot of wasted extra space from backing up the same files over and over.
• Box
  • An alternative to Dropbox, et al.
  • 10GiB free space, 20GiB if you install the mobile app.
• Syncthing
  • Computer-to-computer syncing (no cloud)
  • Supports macOS, Windows, Linux, Android
  • I think iOS is in the works

[bbech]

Can someone point me to the 8TB hard drive that costs $100?

[JohnAtl]

Close ($120). (Not an endorsement on my part, as I’m not familiar with the brand.)

Amazon.com: Avolusion HDDGear 8TB USB 3.0 External PS4 Gaming Hard Drive (PS4...

[oldblueday]

Do I need to back-up up everything stored on the cloud?

So I’ve been keeping all my data in the cloud spread across DropBox and iCloud. This lets me access everything anywhere and on various devices. Also if I replace a device, it’s a simple matter of getting up and running with everything available. Also, I don’t need a large internal SSD.

Now I realize that this is not a backup solution (as a deletion in the cloud is felt everywhere), but what is the purpose of a backup of a Mac with everything in the cloud? There’s nothing here to be backed up. My TimeMachine is probably just spinning its wheels.

Do I need to back up Dropbox and iCloud despite their minimal backups.

Backing up the iCloud Photos Library

Also, the thing I’d care most about is my iCloud Photo Library. This will not fit on my SSD. How can I back this up? I’d still like to keep it all on iCloud.

Advice for backing up iCloud Photo Library

Hi All - I’d love some advice on backing up my photo library that is currently on iCloud Photo Library. My MacBook Pro SSD is unfortunately too small to hold all my originals, and because it is a laptop it is neither always on nor always plugged in. I’ve read a lot of the other posts on the forums here (for example here). But there are some conflicting pieces of advice, so I wanted to follow up. My first idea was to get a Synology and move my photo library to the Synology, but Apple’s KB on t…

Looking at this post, I’d have to

1. buy a big enough SSD,
2. move my iCloud Library to that,
3. delete the iCloud library on my Mac, then
4. let it download everything to the SSD (now I have a backup)
5. remove the SSD (store away safely, make a copy)
6. allow iCloud to reconnect to Photos

Now when I tried this, it said “are you sure you want to delete your iCloud library” and I quickly lost the fortitude to do this. It’s a big library and there are irreplaceable pictures.

Is this the only way?

[WayneG]

@oldblueday

Do I need to back-up up everything stored on the cloud?

IMO, you need to keep two copies, preferably three, of everything you consider important. Like you I keep most of my data in the cloud for many of the same reasons. But I also keep a copy of everything on my Mac so it can be backed up to Backblaze B2.

If you are using “Optimize Mac Storage” and keeping all your files in iCloud you have no backup. Only files that have been downloaded to your Mac can be backed up.

My MBA has a 256GB drive which is not large enough to hold all my data so I store my photos, music, movies, and most of my data on a Samsung external SSD. I prefer Google Drive which allows me to sync to my external drive unlike iCloud which can only sync to my internal drive.

I would purchase an external SSD and move your photos, etc. to that drive. But when you do, don’t delete your photos until your photo library finishes syncing to iCloud and Time Machine completes a backup or two of your new external drive. Confirm that everything is good THEN remove it from your internal drive.

As I mentioned iCloud can only sync to your internal drive and the same is true for Mail, Calendar, Reminders, Notes, etc. So if you start running out of internal storage I would move any other files that you don’t need when away from home to your external drive.

Finally, if you aren’t backing up everything to a cloud solution like Backblaze, in addition to Time Machine, I suggest you add that as soon as possible.

[zkarj]

@WayneG

If you are using “Optimize Mac Storage” and keeping all your files in iCloud you have no backup. Only files that have been downloaded to your Mac can be backed up.

I could swear I heard someone (Craig Federighi?) recently say that APFS and/or the new style of kernel extensions could “provide the flexibility to download cloud files on demand for things like backups”. Am I imagining this?

[Rob_Polding]

BackBlaze only backs up the files physically on your hard disk. I’m not sure with other services but I once set up a Mac and forgot to download all my Photos and it only backed up the photos that were local. I had to set it to download all the photos to the computer before they were backed up.

[Lars]

@SpivR

Long term issues with SSD storage?..

As long as HDDs have a way better GB/$-ratio, I am not even considering SSDs for backups/storage. And since backups run in the background, speed is not an issue for me.

For “bank vault”: I stopped depositing HDDs in a “safe place”. With current bandwidths, I prefer off-site backups. I use both Backblaze and I have access to a remote server.

[WayneG]

I’m not aware of any changes to macOS that would fix this problem. AFAIK the situation is still as @Rob_Polding describes.

[karlnyhus]

Lots of good questions to think about. One quibble, the DiskFresh app may be a good idea but its website looks really old and seems to be Windows-only. Anybody know a Mac utility with similar functionality?

[Shruggie]

Yeah, I think there’s some new API that allow something like Dropbox to be notified “hey, the user is asking for the badgers.jpg file, which I don’t seem to have, so you had better fetch that right quick or I’m gonna be blamed!” by Macos.
No idea if it works - I always keep everything store on device on my computers, though I do appreciate not having to do so on my phone.

[JohnAtl]

@SpivR

I’ve been reluctant to stop using rotational drives for Carbon Copy Cloner backups that I keep offline in a bank vault.

Just a thought: when you’re (royal ‘you’) making a long-term backup, you’ll want to ensure that you’re doing a complete backup that writes everything to the drive. If you’re making an incremental backup, some portion of the data will not be written again, thus it can still degrade over time.

In different words, if you have a backup that you made 5 years ago, and have been doing incremental backups since then, parts of the data on the drive were written 5 years ago, and have been fading since.

Doing a full backup like this negates the need for DiskFresh type utilities to read and rewrite the data.

AFA SSDs, the Wikipedia page is pretty good. One of the interesting things is that Intel’s Optane SSDs use a different technology (resistive, rather than capacitive change to store info). However, there doesn’t seem to be definitive longevity data on either.

[karlnyhus]

@karlnyhus

… the DiskFresh app may be a good idea but … seems to be Windows-only

Looking around for Mac repair utilities, I haven’t found any that specifically attempt to cope with existing data that may be fading on disk. For general Mac repair software, I see little mention of Apple Silicon Macs nor APFS disk drives. Apple gets the blame for providing insufficient APFS documentation and for APFS still being a work in progress.

[cornchip]

Backblaze drive shipments are great. My little anecdotal advice is to do a test request ahead of when you might really need one, if you have a lot of files. Drive provisioning time scaled by number of files more than by total size, and I had ~25x the average number of files on the drive. I made some changes to the backup config in the interest of recovery speed, needless to say…

[James1]

Great discussion, really helpful episode.

You’ve inspired me to upgrade my external drives to SSD @MacSparky.

Do you (or anyone else!) have recommended resources for how to connect a drive (e.g. the Samsung T7) to a network so it can be accessed over wi-fi? I’ve had a look around and can’t find much, any pointers would be really appreciated.

Thanks

[zkarj]

I’m still not sure who I heard mention it (but still think it was Federighi) nor where it was mentioned, but I have tracked down the tech behind what I was referring to (and somewhat confirmed by @Shruggie).

The new technology is called File Provider Extensions and was introduced at WWDC 2021 (so no excuses for DropBox or Microsoft).

Some key passages from the transcript:

The root is what we call a dataless directory. It’s a new kind of object in APFS, and there are APIs to recognize them and interact with them. But more importantly, dataless objects are fully transparent to processes who happen upon them unprepared. Reads trigger downloads, …

When the kernel detects a read access to a dataless file, that syscall is paused while your extension is called to fetch the contents of the file. Your extension’s fetchContents method is called. Typically, you will implement it to perform a download. When the download completes, it calls a completion handler. The contents of the file are handed to the system which fills in the formerly dataless file without invalidating the open file descriptor. The system then unpauses the read access. Now that the file is no longer dataless, subsequent reads won’t have to involve your extension.

The system will evict local files automatically and without involving your file provider extension when there is an urgent need for disk space.

While you’re not involved in disk-pressure triggered eviction, there are methods to trigger or prevent eviction from your extension.

What isn’t clear to me from the transcript is whether a third party, such as a backup tool, could similarly trigger eviction, or whether the File Provider Extension could itself determine that the file was only needed temporarily. But that aside, the system absolutely allows for the download of files when demanded simply by an attempt to read. That much, at least, favours backup solutions. It’s just a case of whether that will start filling your disk.

In fact, if a system like DropBox explicitly allows marking files as “Offline” this, alone, would be information enough to trigger the eviction at some point.