It looks like the Photography Plan still includes Lightroom, LR Classic, and Photoshop for $10/month USD.
Do you use Backblaze B2 for any long-term archival storage?
I don’t, but I would have to if I went the Synology route. I could try this, but because it’s third party, it’s not a long-term solution.
1 Like
Sorry, it appears I was looking at the 1TB plan and not the traditionally priced 20GB plan! Oops!
Doing something that I (almost) never do, and jumping in with a reply before I have read the entire thread. That’s because this is a very interesting thread, but I am at work, and I want to take the time to read it carefully, but also wanted to drop my thoughts on the OP before I forget them…
In any case, one thing to keep in mind is whether your storage needs are easily divided into the things you need immediately available, eg those that really need to be kept on the internal storage of your laptop, and those that are not always needed or at least can be accessed with some delay, which can then be on a server and/or external drive. Assuming you can make this distinction, then some sort of off-laptop storage makes sense. Otherwise, sadly, you need a new laptop with more internal storage and the whole discussion is moot (for you).
I think any solution also needs to address method of access, and method of backup, including local and cloud, as I expect others have already said.
I am currently working off a MBPro (M2) with 2TB of internal storage. It’s enough for everything that I need on a regular basis, including my 1GB photo library, but I do not have any of my media (about 1TB, movies and music) on it, and I probably have another 0.5-1TB of archival storage, also not kept on the internal drive.
I have been running a Synology 4-drive unit for some time. Setting up a Synology is (relatively) easy, although it’s definitely worth finding some articles only or help here in some the early decision making (eg whether to have one drive as a hot spare, whether to use Synology Hybrid Raid or traditional RAID configurations, etc). The nice thing with the Synology is that it is fairly easy to set up SynologyDrive (personal/private Dropbox equivalent) and other servers such as Plex and web servers, which I have running.
One big problem that I face with Synology is offsite backup. There are packages that allow you to make backups to various cloud storage providers (Amazon S3 or Glacier; BackBlaze B2, and so forth), but you are paying by the TB for storage so it can start to run into some costs, and I am not the biggest fan of Synology’s backup software compared with other options including Arq.
What I decided to do recently is to get the cheapest possible Mac mini - the base M2 model, 8GB RAM and 256GB SSD, which I got on sale for $499. I already had 4 x 4TB server-grade drives with very limited hours on them, and so I bought an external enclosure on Amazon and set them up with SoftRAID in a RAID 5 array on the Mini.
Everything on my laptop is kept in sync with the Synology via SynologyDrive and then also synced to the Mini, and the shares on the Synology that are NOT on my laptop (media, archives, etc) are cloned via CCC to the external drive on the Mini as well.
I transferred my BackBlaze license from the laptop to the Mini, and so now for one flat fee everything backs up to BackBlaze. I will extent the storage license to at least one year (extra costs) just in case. I have had the experience in the past of discovering, 6 months after the fact, that something was accidentally deleted and recovering from backups, so I am a bit paranoid about data loss.
As a result, everything on my laptop which is all of my most important data, is replicated in real time to both the Synology and the Mac mini, so I have three readily accessible copies. All additional data on the Synology is also replicated to the Mini, so there are two copies of that on site. Everything from the Mini is in Backblaze, so there is an offsite copy of everything there.
In addition, the laptop does both Time Machine to the Synology and a clone to the Synology every day, so there are two additional copies on the Synology, so that if something goes crazy with SynologyDrive, I have another layer of protection again data loss, because a sync service is not truly a backup, because accidental deletions propagate to the other sync’d computers (although SynologyDrive does all recovery of deleted files, its far easy to find them in the TM backup or the clones, and I have CCC set to protect deleted and change files as well).
1 Like
I wish! I’ve got a lot of images stored up from my years of wedding photography, then product photography (related to web design), then event photography (bundling it with web design). So there’s a lot of media there.
Holy smokes. I think this is the answer for me, folks.
This gets me everything I want: Synology’s ability to hot-swap drives, SMART monitoring with Synology’s email-based alert system, tons of storage availability, and the option to back everything up to another machine that can send it all to Backblaze for cheap. B2 was going to cost me hundreds/dollars a year, so this is going to be a huge cost savings for me.
The only other solution I could come up with, as far as using Synology, was to buy two Synologies, place them in two locations, and sync them to each other. But this Mac Mini solution is a much better way to go, as far as solving the problem: offsite backups.
@nlippman one question for you: what do you do in the case of catastrophic drive failure on the Synology? Let’s say you’re on vacation, away from the house, and multiple drives in the Synology fail at once. Presumably, your CCC schedule continues anyway, and now the data loss is copied to your Mac Mini’s RAID. Is Backblaze your one line of defence in this scenario? (It’s way better than nothing, but just asking.)
@webwalrus thanks again for all your input too. I was very close to pulling the trigger on the QNAP, but I was dismayed it didn’t include SMART monitoring, hot-swappable drives, or email notifications for drive failures. But I think your solution is going to be invaluable for a lot of folks with non-business needs.
1 Like
Great discussion here so far! A few more things.
Firstly, Adobe is very clear in their recommendation that Lightroom (Classic) catalogues be stores on a locally attached drive (internal or external, but NOT networked). The actual image files can, however, be located on a server share - but remember that with today’s sensors generating huge image files, performance will be impacted. Lightroom does allow editing based on the thumbnails stored with the catalogue even if the image file is not accessible (eg your store your image files on a server but are editing on a laptop that is not on the network with / able to access your server), but you CANNOT export final images unless the image file itself is available (which does make sense).
@snelly In the event of catastrophic drive failure on the Synology (eg not just one drive failing and having the RAID array go into a degraded state, which has happened and I have had to replace drives, which is the point of the RAID array in the first place), then presumable SynologyDrive would not be working and clones to/from the Synology would not work, and so copies stored elsewhere would not be affected. I would, of course, lose the clones and TM backups of my laptop stored on the Synology, but presumably I could restore from the copies on the Mac mini and/or BackBlaze, plus the versioning on BackBlaze would hopefully come to the rescue.
A more worrisome catastrophic event would be ransomware taking out the Synology and simultaneously the Mini or the laptop. This is a situation in which a disconnected hard drive stored offsite and updated every can be highly advantageous and is not at all a bad idea, especially with the relatively low cost of external drives. I do have a drive sled as @tomalmy referred to, and since I have some drives sitting around, maybe it’s time to set up a periodic clone as well.
A comment on software vs hardware RAID. There are obviously pros and cons. I decided to go with software RAID for the following reason. If a RAID drive enclosure dies, to recover the data on the drives you are likely looking at purchasing a new enclosure from at the same vendor, quite likely the same model - which might not be available at a reasonable price, or even at all. (This was the issue way back when with Drobo. Because their model stored data in a closed and proprietary format, if your Drobo enclosure died, you bought another one from Drobo and hoped for the best [there were reports that if the enclosure you had was no longer available and you got a newer model, sometimes things did not work]). There were third part data recovery services that claimed to be able to get data from a Drobo drive set. I don’t know how well they worked, but I suspect they were very expensive.
WIth software RAID, if the computer dies, you could (at least in theory) install the software on a different compatible Mac and pull the data from the enclosure. I say in theory as I have (fortunately) not had to test this out.
I am not totally sure what Synology uses underneath to effect its RAID setup, but unless you are using SHR, eg if you use a standard RAID configuration, it might be possible to connect drives from a Synology to another (Linux) computer and use Linux tools to extract data. At one time I was a Linux user and was very familiar with the tools needed to do this, and while that is no longer the case for me, I am sure with some online help I could figure it out if needed.
Obviously this to me is where the backups of backups come in. If my Synology dies, I would be more likely, if I replaced it, to just copy data back from the Mini or BackBlaze, than to try to get data from the raw drives. In fact, when I set up a new Mac, I just set up SynologyDrive with the several shares that I have and let it sit for some hours while all the data syncs up, and I am set. (Well, I also install the software I use, but that’s another story).
QNAP monitors SMART status. It’s right here on the screen in my software. I don’t know that it does email alerts, but the software shows right on the screen the status of your hard drives.
I’m confused. If you’re setting up a RAID 5 array like @nlippman I’m not sure how or why you’d be hot-swapping anything in that external enclosure unless there’s a drive failure.
Once you set up a RAID array, the disks are functionally “glued together” as a single volume. You can’t just rip out a disk and drop in another one without rebuilding the RAID pool - which seems like a Really Bad Idea to me.
Same with the QNAP. The drives are hot-swappable, but you just don’t do that with a RAID array. My 4 8TB drives in RAID 5 on my QNAP are one disk, functionally. If I ripped out a drive it would cause at least temporary chaos not because I’m hot-swapping but because the array would have to rebuild that missing drive on whatever drive I swapped in.
If you just want a box to plug external drives into, and want to be able to flip them in and out at will, any external enclosure will work - but you don’t want to be using a RAID solution.
SHR (Synology’s sauce on top of RAID 5, basically) will let you hot swap without issue as time goes on. It’s a big part of the attraction.
Edit: and for the backup of the Synology, I don’t need any of that. Just a big dumb drive is fine. But the immediate archive bay needs to be flexible for me.
…but you only get SHR as part of DiskStation Manager, which only runs on actual Synology devices, at which point you’re running a NAS with no Backblaze…right?
The thing that it sounded like you were planning to do was this:
which doesn’t seem to be SHR, but rather SoftRAID - which is an OWC thing. And it’s not anywhere close to as flexible as SHR.
No, what I’d like to do is this:
- Synology with DSM running as main backups and archive storage.
- Mac Mini with CCC on the network. It clones the archive storage on the Synology to its own external drive (enclosure irrelevant).
- Backblaze on the Mac Mini to backup the archive to the cloud.
So I have something close to a 3, 2, 1 backup solution for my archive for one low Backblaze fee.
Interesting.
So basically your external drive on the Mini is always going to have to be big enough to store the entire dataset from the Synology?
Sure. Seems easy enough to maintain. Starting with 8TB or so of backup, and scaling from there as needed.
Ah, okay. If you don’t have that much data, and a single drive that will hold it is available, that makes sense. That’s not the case for me. 
I mean, I could always get a QNAP DAS or OWC RAID later for bulk DAS in step 2. Step 2 can scale as gracefully as step 1, as need be, and it runs off Backblaze for a low cost too. Feels like a win.
A few points of (possible) clarification.
Firstly, as @webwalrus points out, you don’t hot swap drives in a RAID system unless you are replacing a drive, either due to drive failure or to increase drive sizes. Drives in a RAID array are not individually useful.
SHR is basically Synology’s way of allowing you to use drives of different sizes in the equivalent of a RAID level 5 array. RAID 5 basically takes n drives (n ≥ 3) and stores data on (n-1) drives and parity information on drive n, so that if any one drive is lost, the full set of data can be recreated from the remaining (n-1) drives. (Subtle point: most raid systems split the parity information as well, rather than one drive being designated to only hold parity information. That does not change anything from our standpoint.)
The thin is that because every item of data on one drive must be pared with data or parity on the same location on the other drives in the array, all of the drives must have the same storage capacity.
What happens if you have two drives that are 8TB and 2 that are 6TB? Then 6TB are used in all 4 drives, and 2TB on each of the 8TB drives cannot be used for anything in that raid array.
You could use the 2TB left on those two drives to make another volume, using RAID 0 or 1, or any raid level allowed for the number of drives with excess capacity.
If you upgraded the two 6TB drives to 8TB, then there is 2TB free on each drive, and that could than be added to use all 8TB of each drive in one array.
What SHR does is uses the leftover space to create some form of RAID (depending on how many drives there are with leftover space) and, in software, adds it to the base RAID5 array so you don’t lose the use of that space.
You will note that if you had 3 x 6TB and 1 x 8TB, the leftover 2TB on that 8TB drive remains unused because you have no other location to create redundancy.
SHR does all of this magically, so that if you had 4 x 6 TB in your SHR and one at a time upgraded each to 8TB, you will wind up with a 4 x 8 TB SHR volume at the end of the process.
SoftRAID does not support any of this magic. Instead, if you had 3 x 8TB drives and 1 x 6TB, you would create a 4 x 6TB RAID 5 volume. You could, if you so wanted, use the remaining 2TB in the three drives to create a 3 x 2 TB RAID 5 second volume, but that would NOT be part of the first volume as it would have been under SHR. If you did not do that, then once you upgraded the 4th drive to 8TB, you could then add the extra 2TB from each drive to the underlying RAID 5 and expand the RAID 5 volume. It’s a bit more complicated than doing all of this with Synology, and I’ve never done it, so I don’t know the exact sequence of steps. I am sure they have a manual or some online instructions for this.
As an aside: If you are planning a Synology as the NAS and the Mini with external storage as a copy of the NAS data providing the BackBlaze option, you might want to just to with a large single external drive on the Mini. Yes, it’s a single drive failure risk, compared with RAID where you can survive a single drive failure, but it’s functioning a one of your backups, and if your data is on the laptop, the Synology, the Mini, and BackBlaze (which is my setup for active data) or Synology + Mini + BackBlaze (which is my setup for the other data not kept on the laptop), you probably have sufficient redundancy without the extra overhead/hassle of RAID on the mini at lower cost.
I did the external RAID on the mini because I already had the drives, the enclosure was low cost, and the SoftRAID folks (at OWC) gave me an upgrade price as I had a license for an older version of SoftRAID. It would still have been cheaper to just buy an 8 or 12 TB external drive, but then I would have felt bad about disposing of perfectly functional smaller hard drives and I had no other use for them.
2 Likes
@nlippman i think this makes sense. If the Synology has 5 4TB drives in SHR, and I wanted to double it, could I replace one drive at a time and let it rebuild?
@nlippman sorry, final question: does CCC care that the Synology is not HFS+ when it’s copying data from the Synology to your DAS on the Mac Mini?
That is correct. Remember that you must do the rebuilt after each drive swap. DO NOT remove more than one drive at a time or your RAID array will be damaged.
1 Like
It does not. CCC sees each share on the Synology as an SMB-mounted network volume. In general, MacOS is reasonably good about preserving filesystem metadata on network shares, by storing the metadata (tags, for instance) in a sidecar file.
As long as you are dealing with data volumes, CCC will work fine. The place where you get into trouble is if you want to clone your system volume, because you cannot do so to networked shares. To do that, you would create a sparse bundle on the Synology and the clone would be done into the sparse bundle. I do not do that because
a) I don’t attempt to clone the system volume. Since Apple made it a major deal to try to create a bootable clone on an external drive, the utility of cloning the system volume has gone down. If I need to rebuilt a Mac system, I would just reinstall all of the software, attach my data, and deal with the hassle of recreating preferences. It’s a pain, but there it is.
b) If you clone into a sparse bundle, you can format the SB as APFS or HFS+ as you like, but then the contents are only accessible by mounting the sparse bundle onto your Mac. If you just use a folder on the Synology, all of the files are browsable directly on the Synology as well, albeit with a bit more potential issues with weird characters in file names and the metadata. I chose that over dealing with another layer of abstraction in the filesystem.
1 Like