Software to sync a larger disk image between 2 macs

I have a larger disk image (sparsebundle) that I want to incrementally sync between 2 macs. iCloud can’t do block level syncing so if one thing in this disk image changes, it would re-upload the entire thing.

Dropbox can do block-level syncing, but I don’t want to add another monthly subscription just for syncing this one file.

Is there other software that can be used for this? Maybe even just on the local network as a workaround?

This article (How to sync, share and transfer files from Mac to Mac) on SetApp recommends ChronoSync but I’ve never heard of it before either.

1 Like

ChronoSync has been around for years, and is a Mac stalwart. Recommended.

However if you are ok with the terminal, the rsync command will do the same thing.
Indeed, rsync stands for “remote sync”, and in many ways ChronoSync is a GUI
front end for rsync, with additional functionality added.

A simple search will give you examples of the rsync syntax you could use.

(I’d search for you, but the command syntax is specific to YOUR use case)

3 Likes

rsync isn’t block level sadly, so it would also just reupload the entire fileblob instead of incremental changes whenever the bundle changes

1 Like

I’ve discovered GoodSync (File Sync & Backup Software | GoodSync) which looks almost perfect. It has heaps of features to sync between different services, local network and local folders, even on the block level.

But, it’s also a $30/year subscription

There is a free tier that looks sufficient, but it has a 100 files limit on “Number of files”

If the bundle is treated as one file it should work, but if it’s treating each of the bands inside that bundle as separate file I wouldn’t be able to even sync a fraction of this image with it

I believe you can force the block-level algorithm to run
by including the --no-whole-file flag. However I am not in
front of my Mac to do a man rsync and see if it’s there

Curious as to why you would want block level, are you
concerned about speed of sync?

1 Like

Okay to give some background what I’m trying to do, I’m trying to synchronize the database of a piece of software I’m using between 2 macs. I tried the “symlink into iCloud Drive” route, but it didn’t end well.

You can tell the software to use a database off an external drive so you can have the entire DB integrity packaged away. When mounting a sparseimage or disk image, it’s detecting it as external drive and allows you to move the entire database onto that bundle. The entire bundle is currently 30gb large, growing slowly as I’m adding more stuff to it.

If I don’t have block level diffing, the cloud software (like iCloud Drive) treats every little change inside that bundle as if the entire bundle was a new file and re-uploads the 30gb chunk every time. That makes using it across 2 machines very awkward

It looks like you’re right and rsync does support delta-transfers - diff - Can rsync update a large file that has only changed partially without full retransmission? - Unix & Linux Stack Exchange

I’ll take a look and see if this works for my usecase!

Also I think a -S (or is it lower case?, sorry) will
handle sparse images. Might address the band
issue with the db.

All other things aside, a sparse bundle (as opposed to a sparse image) should only sync the bands that change and not the whole thing.

I’ve just tried this and uploaded the entire sparsebundle onto iCloud Drive. Then mounted it and added an empty folder. The entire 30gb sparsebundle is now marked as ‘waiting for upload’

So iCloud Drive at least can’t handle syncing individual bands only

That’s really interesting to me. My understanding is that the bundle really is just a directory and the bands are just files. I’ll have to investigate this too :slight_smile:

After considering my options and trying both ChronoSync and GoodSync I ended up giving up on the sparsebundle idea completely.

If I can already tell Chronosync/GoodSync to sync a specific folder 2-ways, there is no need in even using a sparsebundle.

Are there any other syncing solutions that allow me to sync and properly diff folders against a cloud version (like s3)?

GoodSync vs Chronosync, I feel that GoodSync is more powerful and has more options than ChronoSync but the subscription is kinda a deal breaker

Hm. S3 kind of complicates things (you could also use Backblaze’s B2).
To sync folders between computers, I use ResilioSync. You could then have one of those computers backing up to S3/B2 at some interval - maybe once a day. For that I would use Arq. There are also utilities that allow you to mount S3 like a drive, such as Fuse, Transmit, or CloudMounter. Amazon doesn’t recommend using S3 as a block file system though. Personally, I’d just backup to it with Arq once a day.

I’m using arq for backups to s3 as well!
Let me look into ResilioSync, it looks interesting

When using ResilioSync for p2p, you reckon it’s better to sync the sparsebundle between the machines, or the content of that sparsebundle?

I just need something to make sure that this actively used app database is replicated correctly without corruption, and conflicts are handled properly. There are a bunch of smaller files in there as well that the app needs to function, so the idea with the sparsebundle was to isolate everything into that image, and have that synced properly as one thing, instead of a lot of smaller things.

Dropbox at least has a rock solid syncing algorithm, but yeah I am not a subscriber currently

If anyone has other ideas, let me know!

I would sync the Sparsebundle, but only if it unmounted regularly, otherwise I don’t think Resilo would sync whilst the bundle is mounted. I could be wrong in that regard.

If you did the content of the Sparsebundle, you’d need both machines on, with both bundles mounted.

Like GoodSync and Chronosync, as you could sync two folders directly, you may not want to bother with the sparsebundle?

I tried Resilio Sync and was able to sync the 20gb+ sparsebundle to my other machine in like a few minutes, I’m impressed!

I use that as my interim solution until I can figure out how feasible it would be to sync the folders directly without sparsebundle. For now it’s all nicely encapsulated into the bundle, and identical on both machines

1 Like

All right I wanted to buy a resilio license but noticed that the app has been inactive for a long long time (almost 2 years). People on reddit are warning about the state of the app and that it is likely abandoned.

Looking for alternatives to resilio. Any good ones besides Syncthing (giving that a try now)? Preferably with a macOS app