A cautionary tale about iCloud Photos

Software
, ,
1

After holding off of iCloud Photos for a long time, I decided a few months ago to finally make the jump, mainly due to two factors. On the one hand, the changes to Photos in iOS 13 were compelling. Additionally, at that time, I was on the 200GB plan which was starting to get just a bit too small for my document storage needs. So I upgraded to the 2TB plan which then basically gave me iCloud photos ‘for free’. Shortly after, I made an additional backup of my library and then checked the box to activate iCloud Photos on my MacBook Pro where all of my photos were stored. That’s were a month-long journey started…

Once the upload was finished after a couple of days, it told me that about 1600 of my 40,000 photos could not be uploaded to iCloud. At that point, I was not very concerned and tried some general troubleshooting (eg. restarting photos.app, restarting the Mac, giving it some additional time while plugged into power). However, nothing changed. So I decided to deactivate iCloud Photos on the Mac, went into the iCloud preferences and deleted all my pictures from iCloud. As a safety measure, iCloud deletes pictures after 30 days, so I had to wait.

After the 30 days, I tried it again. Unfortunately, the result was the same - still about 1600 photos could not be uploaded. I then investigated whether there was a particular picture file format or camera was causing the problem. But the pictures were a mix of HEICs, JPGs and RAWs from different iPhones and cameras. Of course, as I was about to do a serious change in my library, I made another backup. Then I manually exported the pictures as originals out of my library and imported them back in. After importing them again, they were uploaded without any problems. I did cost me some time as I have all my pictures sorted into albums which meant that I had to sort them all again but I was happy that I could finally enable it on my other devices.

To avoid further trouble, I first imported any remaining pictures from my iPhone and iPad to the Mac the old fashioned way via a cable and only activated iCloud Photos after deleting every photo left on the device (and also removed it from the ‘recently deleted’ folder). I also did this one device at a time and waited until Photos was done downloading thumbnails). A couple days later, my Photos were successfully synced to my iPhone, iPad and Mac mini - or so I thought…

I noticed the next day that the count of pictures and videos differed an each device:

Device No. of all items No. of photos No. of videos Notes
Mac mini 40,329 39,312 1,017
MacBook 40,333 39,316 1,017
iCloud.com 40,327 39,309 1,017 '1 other element'
iPad 40,330 39,315 1,015
iPhone 40,331 39,316 1,015

In the process, my MacBook told me that there wasn’t enough space on my disk to keep all originals (which is in itself weird, as I had about 40GB left on my drive and before uploading the pictures, all pictures were solely on the MacBook). Since I didn’t see a smart way to proceed, I called up AppleCare and asked them what to do. The support rep told me to deactivate and then reactivate iCloud Photos on each of my devices. I specifically asked him whether that could result in data loss. He told me it was not possible because the pictures that were uploaded would still be in iCloud and every picture that hadn’t been uploaded would remain on device.

When I tried this solution on my iPhone, it told me it would only delete about 26,000 photos (bear in mind that the iPhone was empty before I activated iCloud Photos). I proceeded nevertheless and deleted the remaining 14,000 photos manually. At this point I was fine with loosing a small amount of possibly not uploaded photos if that meant I would get the same count on each device. After reactivating iCloud Photos on the iPhone, nothing had changed - the numbers still did not add up.

Because this was more or less expected (the support rep told me that it could be any of my devices that was causing the problem and turning iCloud Photos off and on again on all of them would probably solve it), I continued on my iPad. After deactivating it, I experienced more or less the same. About 1,500 photos remained on device as well as all albums. I deleted them manually again, reactivated iCloud Photos and went to sleep.

This morning, when I checked whether everything was available again on the iPad, I noticed that about 95% of my albums were missing. I checked all the other devices and realised they were gone everywhere. I suspect what was happening was that in the background, iCloud Photos was not really deactivated, so deleting the albums manually caused them to be deleted everywhere as well.

So, I went and got my backup library from before I solved the problem of pictures not being uploaded and tried to open it. I was greeted by a lovely error message that some pictures in my library were not available locally in the library but only in iCloud. Therefore I was supposed to make this library my main library and then let iCloud download them. But doing so would of course sync changes and delete the albums. (As a side note: At the time of the backup, the library was set to keep originals…).

Okay, no problem, I still had the backup from July before I started the iCloud Photos journey. So I can just restore that one, stay away from iCloud and everything will be fine, right? Well, Photos opened the library (which was still on an external drive) and started to update it (not an iCloud update but the occasional library update) but even after about 15 minutes, it still remained on 0%.

That was when I started to have a bad feeling about this. I had one broken backup and one that was stuck in an update-process. Because this whole process went over months, my TimeMachine backups did not reach far enough back to get a problem-free version from my MacBook. So, I went to my last chance and got my offsite backup (created with Carbon Copy Cloner), booted from it, turned off Wifi immediately and - luckily - the library opened. Unfortunately, I could not copy the library anywhere because it also told me that some pictures in my library were not available locally in the library but only in iCloud. But since the library was opened, I could at least go through all of my albums and take screenshots of them. That way, I am at least able to recreate the albums for about 20 years of photos… I guess, there’s another big project I can add to my task manager…

So, the moral of this story: Even a lot of backups and caution cannot prevent crisis sometimes. My photos are more or less safe but still, this is bad. After years of having no problem at all with iCloud Drive and iCloud sync (I’ve been using document syncing over iCloud in its various states since about 2012), I now understand the occasional criticism of Apple‘s service quality.

A side note:
Having gone through this, I find it baffling that there is no restore functionality for iCloud Photos as there is for documents stored in iCloud.

Some additional info:
Whenever I mention ‘pictures’, I actually mean pictures and videos. For better readability, I chose keep it simpler. (One exception is the table above, where I detailed both pictures and videos)

My iOS (iPhone X, iPad Pro 11“) devices are running 13.1.2 - the latest public version at the time of writing.
My Macs (15“ MBP 2016, Mac mini 2012) are running 10.14.6 (Mohave).

5 Likes
2

It’s quite a read, and I have nothing but sympathy. Synching is hard and – given the number of items to get right – there just seem to be too many pitfalls with it. Obviously the system isn’t robust enough to recover from a stumble here or there … so I echo your wish for Apple to spend some time figuring out restore / repair scenarios.

1 Like
3

Thank you so much for this detailed account. I know it took a long time to write this and an even longer time to go through all this pain. I am very protective of my photos, as I have many from life (don’t we all?) and also my business. I haven’t found an easy solution for keeping them separate. Co-mingling them is not an option.
I have yet to embrace Photos. That said, I am always on the hunt for a storage and management solution. Your story gives me great pause and also a feeling of justification that Photos may not be for me. I sympathize with your pain and thank you again.

2 Likes
4

The macOS and IOS Photos apps are OK, at least for the display and editing of photos on the respective devices. iCloud Photo Library has a fundamental design problem, however. It works by syncing photo and video files among various devices, a process not optimal for backup.

I would prefer an iCloud solution that prioritizes fail-safe backup over syncing among devices. I don’t need immediate access to ALL of my photos on my phone. Access on the phone to some photos, yes - perhaps the last year, plus additional “permanent” photos designated in an “always available” album. More important, in my view, is secure backup of all of my photos. In the current iCloud Photo Library model there are just too many ways that permanent secure backup can fail.

6

To add to Arthurs point (and maybe that’s what he meant anyhow) is that with iCloud photos enabled, there are many ways in which the usual backup solutions can fail. Which is exactly what happened to me. macOS decided that there was not enough storage on my Mac, so it deleted original size photos on my Mac even though “keep originals” was enabled on my Mac. Because of that, my manual backup of the library (which I put on an external drive) could not be opened without syncing to iCloud before which would have deleted the things I tried to restore.

A similar problem would occur with TimeMachine as the pictures macOS deleted off of my Mac would not be available in the latest backup. Therefore, there is a risk of many backup solutions failing when there is a problem with the synchronisation of iCloud photos.

2 Likes
8

I would be interested in knowing how you proceed with this @Wolfie

1 Like
9

Quote from your reply:

Quote from my post:

Due respect, I missed the point of your reply. My description of iCloud Photos appears to be similar to yours.

I simply tried to comment in a tactful manner, considerate of those who value iCloud Photo library for reasons other than strictly-defined backup.

10

I’m thinking of several ways to do this, each with its own set of tradeoffs:

Option 1a
Export all photos as unedited originals to an external drive. When checking the “Export IPTC as XMP” box, you’ll preserve metadata like changed day and time or location which was added in Photos. However, you’ll loose any edits you’ve made on your photos.

Option 1b
Export photos to an external drive. You’ll preserve metadata like changed day and time or location which was added in Photos. You’ll also preserve the final result of any edits you’ve made on your photos. However, the photos will be reduced in quality, for example when there RAWs in your library.

For both 1a and 1b, it might get a bit complicated to remember which photos have already been exported since all of them are in the same folder. Option 2 might help with that but will also take more time.

Option 2a
Export all photos as unedited originals sorted by year to an external drive. In this case, it would be easier to see that for example all photos from 2010 until 2017 have already been copied (assuming that you do not add photos of prior years to your library) because there would be a folder for each year. The same caveats as in option 1a apply.

Option 2b
Export all photos sorted by year to an external drive. In this case, it would be easier to see that for example all photos from 2010 until 2017 have already been copied (assuming that you do not add photos of prior years to your library) because there would be a folder for each year. The same caveats as in option 1b apply.

All solutions presented here have in common that in a worst-case scenario your albums would be lost. But at least, you would have your photos. Of course, you could also export each album separately but this would be a lot more work and if you have photos in multiple albums, this would mean that your backups take up more space as these photos would be exported multiple times.

11

An update to the whole situation:

As I really did not want to rebuild all of my albums, I wanted to try some final ideas. So, I first bought a 5TB external drive and copied all my various backups (remember, most of them were unusable, but it couldn’t hurt to save them) to it. In total, this was about 1.5TB of storage. I then let it all backup to Backblaze to create another level of safety.

I then deleted my iCloud photo library from my MacBook and copied the original but apparently nonfunctional library back onto the MacBook. As a quick reminder, the last time I tried to open it, the following happened:

Okay, no problem, I still had the backup from July before I started the iCloud Photos journey. So I can just restore that one, stay away from iCloud and everything will be fine, right? Well, Photos opened the library (which was still on an external drive) and started to update it (not an iCloud update but the occasional library update) but even after about 15 minutes, it still remained on 0%.

Luckily, with the library back on the internal SSD, this library update went through rather quickly and I was presented with my old library from July - with all my albums. On my Mac mini, I then exported all the newer pictures which resided in the iCloud library and reimported them into my restored library on the MacBook. So luckily, I’m now back with a fully functioning local library with all my photos and albums.

Then, I turned off iCloud Photos on all my devices. Again, my iPhone did only delete about one third of the library. The remaining 20,000 items had to be deleted manually - which is not a joyful thing to do… (One trick to speed this up is to create a new empty Photo library on the Mac and then import all pictures. There you can check the checkbox to delete the pictures from the iPhone after importing them. However, I still had to delete some of them manually.)

Having done that, I now have to wait 30 days for Apple to permanently delete the pictures. Maybe, I’ll start another try to set up iCloud Photos then. But as much as I would like to have the features, having gone through this, I’m not sure whether I really want to deal with it again. If I do, I’ll make sure to make even more backups than I usually do before attempting a big project. If you are planning to do something with iCloud Photos, I can only recommend that you take extra caution, create lots of backups and make sure that they are functioning.

1 Like
12

You don’t have to wait 30 days for auto deletes in iPhoto on a Mac, you can go to the Recently Deleted folder select them all with shift click then right click and delete.

13

That’s correct. However, the way I understand it, after deactivating iCloud Photos on all your devices, Apple keeps the whole library for 30 days before deleting it. Maybe I could go to iCloud.com and delete all pictures there but maybe Apple deletes more stuff behind the scenes after those 30 days. After the whole journey, I try to be especially cautious.

14

OK - now I have an issue. My Mac has “download all photos” set on, so all the photos should reside on my Mac. I am trying to export 13 photos that I have taken this year, none recently. 10 have exported, 3 have not. The little clock icon is stuck and when I click it, I see two progress bars - one says Exporting 10 of 13 items, the other says Downloading 10 of 13 items for export. The download process broke with 3 items to export, although internet connection is working otherwise.

So if they are already stored on my Mac, displayed on my Mac, edited on my Mac, why am I having to download them to export? That’s a rhetorical question. Apple’s cloud first philosophy assumes we all have infinitely fast internet with no data caps.

Further info - I had edited the 3 photos, and their edits are stuck not uploading to iCloud, therefore they can’t be downloaded to export. This is not what I want when I keep originals on my Mac. Rebooted computer, edits got uploaded, export then worked immediately. Sigh.

3 Likes
15

Excellent description of problems with the Photos part of iCloud.

Any comments on the bigger iCloud picture? Various parts of iCloud have been disappointing over the years, starting with the infamous MobileMe. A memorable example of iCloud’s flakiness was seen after an OS X upgrade during my early years of using Dropbox, probably around 2014-2015. Dropbox users who chose to sync the Dropbox Vault using iCloud experienced significant problems, This was sufficiently widespread that during that period 1Password support recommended using another sync method - Dropbox or wi-fi - instead of iCloud sync. That problem was eventually fixed, but various parts of iCloud exhibit problems to this day.

I don’t trust iCloud to be reliable. It’s OK to use for preliminary drafts of Pages or Numbers documents, non-critical Notes, syncing Safari bookmarks and similar uses. For more important records requiring sync among devices, I have much more trust in Dropbox, combined with a true backup system of local + cloud storage (not iCloud). I would not consider syncing my main Documents folder through iCloud.

I would be interested in others’ thoughts …

2 Likes
16

Apple has lost me when it comes to believing in iCloud. “Someday it will work” is not on my calendar. :frowning:

1 Like
17

My feelings as well.

18

I’ve probably tried every service Apple has offered, either because I was interested personally or because I had an “early adopter” senior executive that expected support.

I still have my @mac.com email, but only use it for Apple Store & iTunes receipts, etc. I prefer Gmail.

Safari & Keychain syncing seems to work perfectly every time. And I do sync my photos to iCloud (& Google Photos) but also use Chronosync to copy my original photos to an external drive for backup via Arq.

I keep very little in Notes because there is no way to restore single files deleted more than 30 days. Pages and Numbers are very capable programs, but I am Google Docs spreadsheet user. My “word processor” is BBEdit, or TextEdit for those rare times I need to print a decent looking hard copy.

iCloud could really use a “Sync Now” button, it never syncs immediately like Google Drive or Dropbox. And Apple’s insistence of hiding the actual file structure in ~Library makes relocating these folders to an external drive impossible - which is a non-starter for me.

These days I save files to iCloud when an IOS app doesn’t support Dropbox or Google Drive. And use Chronosync to immediately move those files to GD. (until Hazel & Catalina work things out)

IMO, using iCloud is like having your boss’s kid as your assistant. They can get the job done, but it’s rarely the way you would prefer, and almost never on time. And there is nothing you can do about it. :slight_smile:

4 Likes
19

I trusted contacts syncing and still do. However, nowadays I trust but verify. Once a few years back I entered the address for a place I was going into my Mac and then got in the car with my phone to navigate from the passenger seat. After we took off, I opened the Contacts app - no new entry. Doh!

3 Likes
20

I experienced another significant problem with iCloud Photos, specifically the Photostream function. To avoid hijacking this informative thread, a separate topic was created. Everyone is invited:

https://talk.macpowerusers.com/t/another-cautionary-tale-about-icloud-photos-photostream/14361

2 Likes
21

In general, I’ve never had any problems with iCloud, especially in recent years. I’ve stored most of my documents in iCloud for years and it has been fast and rock solid. I also love the desktop syncing feature which works perfectly for me. With the exception of iCloud Photos (which, admittedly, is a big exception), I have nothing to complain about. But because being save is better than being sorry, my documents folder is backed up to an external drive every 6 hours via Carbon Copy Cloner.
Given Apple’s stance on privacy, I just feel better trusting them with my data than Dropbox, especially with the sketchy stuff they’ve done with their Mac app in recent years.

1 Like
22

I do my own sync myself via my own Server. Yes, it means I can’t sync except for at home but so far not an issue. I use iCloud for safari Bookmarks, Dropbox for Scrivener documents that are not private and Google Drive for things that the general public (or at least a much larger subest of folks) may access.

Everything is also backed up locally.

2 Likes