Photos – fantastic, until it's not... Sync woes

Software
1

So I have Photos set to pull down full-res versions off iCloud, onto my Mac Mini. I do this with both my wife’s library, and mine.
The Mac Mini is then backed-up to a RAID DAS, and via Backblaze. And the occasional “other” Lacie 2TB HDD, just to have another physical copy somewhere.

Recently went in to both libraries, and noticed that no newer photos since end of July had been downloaded, despite Photos (on both accounts) telling me it was updated.

So did what usually fixes things, by deselecting, and then reselecting, iCloud Photos.
Well.

Figured something was up, when it then started re-organising my library, and looking for duplicates/favourites etc. Left it be.
This morning, told that I no longer have sufficient space on my HDD.
Ran a DaisyDisk scan, and all of a sudden, my Photos library has ballooned to twice the size, and taken up all the space.

What appears to be happening is that it started pulling down the entire library again from iCloud.
Presumably(?) once this is done, it will then compare, and remove the duplicates.
Regardless, I don’t quite have 700GBs available on my 2nd internal HDD, for the duplication download, prior to removal (hopefully) of said duplicates…

So now I’m busy copying the library to the Lacie external 2TB HDD (which has had to be reformatted).
I will then let it do its thing on the external, and hopefully get everything back down to its actual, proper size.
And then, I will copy it back to the internal HDD.
Sigh.

I appreciate the fact that it has been working flawlessly for years at a time.
And yes, I did “trigger” things, in an attempt to force the sync again (albeit that it simply stopped for no rhyme or reason), but it’s still frustrating that everything is so opaque.
So yes, it’s fantastic – until it’s not.

/Rant over.

4 Likes
2

Hi Brad,

When you say you do this for both your Wife’s and your own libraries, is this on 2 computers or the same computer, but via 2 different logons?

3

Yes, you apparently triggered the infamous iCloud Photos sync quagmire :grinning:

Allison Sheridan in her blog Podfeet (link below) explains this and provides a clever hack to avoid the iCloud Photos sync hell. Basically, you create a new empty photos library, make that the system library, then sign out of iCloud (or de-select iCloud Photos). Upon re-entering iCloud or iCloud Photos, your full library is downloaded as a fresh copy in your new system photos library.

Why do this? It turns out that downloading the photos library anew is faster (and probably safer) than letting Photos sync a large library because the iCloud service must re-check every single image to see if it has been synced to and from iCloud. Allison claims that this process of checking every image would take about 3 weeks for a 70,000-image library with a 100Mbps internet connection.
Allison Sheridan’s Podfeet link: https://www.podfeet.com/blog/2018/03/photos-sync-4-days/

Allison’s explanation is outstanding … and I am surprised that it is not more widely known or discussed.

1 Like
4

What an outstanding article! Thanks for sharing. I had major photos sync issues this last year and absolutely hated life while trying to figure out how to fix it. This article went into my bookmarks in case I ever have the issue again.

1 Like
5

Same computer (Mini), two separate accounts/log-ins.

6

Thanks for this. Might very well have to go this route.

7

EDIT:

Patience is a virtue. Thank goodness I waited. Successfully moved over, just took really long to get through those last few seconds.
Now to rebuild the new library.

Sigh.

So having decided to go the quicker route of rebuilding my iPhone library, I opted to move my existing library from the Mini, to the external HDD.

Given we are looking at north of 450GBs, was going to take several hours.
At least it was churning away, until now.

But now, it’s been stuck here for the past few hours…

Finder

I guess best to just leave it be for a bit longer?
Finder is not listed as “not responding”…

And since I am going to rebuild the library from scratch (from iCloud), should be fine…
But would prefer to have at least one library intact, before starting up the new…

8

FWIW, I’ve never thought the Finder was very good at moving very large directories. I’ve had better luck copying large (400 GB +) .photolibrary files with rsync.

1 Like
9

Round two in this drawn-out mini-saga…: :roll_eyes:

The rebuilt library seems to have done its thing – so undeniably faster than what would have been the case, had I not started afresh.

The problem?
Finder tells me the “newly rebuilt” library comes in at just over 70GBs.
The original library, that it has now replaced, was 460GB…

I should be worried? Surely I should be worried?
That said, all appears to be there… So no idea why the discrepancy??

Any thoughts?

10
  1. It could be that everything hasn’t finished downloading. Do you have “Store originals” enabled?
  2. I don’t believe that it syncs deleted items. Is it possible that you have a LOT of deleted items in the old library?
  3. Some discrepancy can be when it hasn’t generated thumbnails and previews. But I don’t think that would account for a 390GB difference.

I would be a bit concerned That’s a large difference. :frowning:

When you select files (old ones that you haven’t touched in a long time) are they fully loaded and editable? If so, and you’ve done a large spot check then it might be alright, and I would lean toward it being a bit of #1 & #3.

1 Like