I have a larger disk image (sparsebundle) that I want to incrementally sync between 2 macs. iCloud can’t do block level syncing so if one thing in this disk image changes, it would re-upload the entire thing.
Dropbox can do block-level syncing, but I don’t want to add another monthly subscription just for syncing this one file.
Is there other software that can be used for this? Maybe even just on the local network as a workaround?
ChronoSync has been around for years, and is a Mac stalwart. Recommended.
However if you are ok with the terminal, the rsync command will do the same thing.
Indeed, rsync stands for “remote sync”, and in many ways ChronoSync is a GUI
front end for rsync, with additional functionality added.
A simple search will give you examples of the rsync syntax you could use.
(I’d search for you, but the command syntax is specific to YOUR use case)
I’ve discovered GoodSync (File Sync & Backup Software | GoodSync) which looks almost perfect. It has heaps of features to sync between different services, local network and local folders, even on the block level.
But, it’s also a $30/year subscription
There is a free tier that looks sufficient, but it has a 100 files limit on “Number of files”
If the bundle is treated as one file it should work, but if it’s treating each of the bands inside that bundle as separate file I wouldn’t be able to even sync a fraction of this image with it
I believe you can force the block-level algorithm to run
by including the --no-whole-file flag. However I am not in
front of my Mac to do a man rsync and see if it’s there
Curious as to why you would want block level, are you
concerned about speed of sync?
Okay to give some background what I’m trying to do, I’m trying to synchronize the database of a piece of software I’m using between 2 macs. I tried the “symlink into iCloud Drive” route, but it didn’t end well.
You can tell the software to use a database off an external drive so you can have the entire DB integrity packaged away. When mounting a sparseimage or disk image, it’s detecting it as external drive and allows you to move the entire database onto that bundle. The entire bundle is currently 30gb large, growing slowly as I’m adding more stuff to it.
If I don’t have block level diffing, the cloud software (like iCloud Drive) treats every little change inside that bundle as if the entire bundle was a new file and re-uploads the 30gb chunk every time. That makes using it across 2 machines very awkward
I’ve just tried this and uploaded the entire sparsebundle onto iCloud Drive. Then mounted it and added an empty folder. The entire 30gb sparsebundle is now marked as ‘waiting for upload’
So iCloud Drive at least can’t handle syncing individual bands only
That’s really interesting to me. My understanding is that the bundle really is just a directory and the bands are just files. I’ll have to investigate this too
Hm. S3 kind of complicates things (you could also use Backblaze’s B2).
To sync folders between computers, I use ResilioSync. You could then have one of those computers backing up to S3/B2 at some interval - maybe once a day. For that I would use Arq. There are also utilities that allow you to mount S3 like a drive, such as Fuse, Transmit, or CloudMounter. Amazon doesn’t recommend using S3 as a block file system though. Personally, I’d just backup to it with Arq once a day.
I’m using arq for backups to s3 as well!
Let me look into ResilioSync, it looks interesting
When using ResilioSync for p2p, you reckon it’s better to sync the sparsebundle between the machines, or the content of that sparsebundle?
I just need something to make sure that this actively used app database is replicated correctly without corruption, and conflicts are handled properly. There are a bunch of smaller files in there as well that the app needs to function, so the idea with the sparsebundle was to isolate everything into that image, and have that synced properly as one thing, instead of a lot of smaller things.
Dropbox at least has a rock solid syncing algorithm, but yeah I am not a subscriber currently
I would sync the Sparsebundle, but only if it unmounted regularly, otherwise I don’t think Resilo would sync whilst the bundle is mounted. I could be wrong in that regard.
If you did the content of the Sparsebundle, you’d need both machines on, with both bundles mounted.
Like GoodSync and Chronosync, as you could sync two folders directly, you may not want to bother with the sparsebundle?
I tried Resilio Sync and was able to sync the 20gb+ sparsebundle to my other machine in like a few minutes, I’m impressed!
I use that as my interim solution until I can figure out how feasible it would be to sync the folders directly without sparsebundle. For now it’s all nicely encapsulated into the bundle, and identical on both machines
All right I wanted to buy a resilio license but noticed that the app has been inactive for a long long time (almost 2 years). People on reddit are warning about the state of the app and that it is likely abandoned.
Looking for alternatives to resilio. Any good ones besides Syncthing (giving that a try now)? Preferably with a macOS app