The case of disappearing iCloud Files

I have my Documents folder store in iCloud. Inside this Document folder, I have subfolders like Personal, Developers, Kids, Assets, etc. I am aware of “Keep Downloaded” features in Sequoia and I have done this for a few important folders like Personal and Kids. I noticed a few months ago that some of my source files in Developers were missing. I went to Time Machine and found them and retrieve them from Time Machine. Back then, I wondered how these files could go missing. Not all of them, just a few like Xcode project file (but not the .swift files).

Today, my daughter her Piano certificate and I want to scan and store them in Kids. I looked at the Certificates folder and all my hardwork of scanning their certificates were gone! I was positive I have scanned them. Again, I looked at Time Machine and went back to one year ago and sure enough, the files are there! They are right there!

I don’t want to think that iCloud is deleting my files but I don’t think I’d have inadvertently deleted them. I understand if iCloud takes them off my HDD (to optimize Mac storage), and if so, I should see the files in the folder, and there should be an icon to show it is in iCloud. But I am NOT SEEING any files. Just the folder Certificates with no files inside them. In this case, the files are not shown at all, that means it was deleted! I do have Optimize Mac Storage turned on.

Yes, I paid 2TB subscription every month and still have 800GB of free space!
But if iCloud is somehow silently deleting my files, I cannot depend on this.
Am I going nuts? None of you have seen this happening at all???

1 Like

If they are there in iCloud in Time Machine and they are not there now, then clearly something deleted them.

I think I’ve experienced something similar before, but chalked it up to user error or some weird app interaction. I’m keen to follow this thread though!

There are stories “out there” about iCloud’s unreliability for files … for some Apple’s iCloud is perfect, for others, not so much. I don’t use it very much but for simple synching of Apple apps which seems to work. Just a few things to check which may or may not help.

  • is your computer set to “Optimise Disk Space”, and if so iCloud will delete stuff on your computer to save you space. Isn’t that nice of them. :sweat_smile: :rofl:
  • check the iCloud web site and see if files there. www.icloud.com

My computer is set to Optimize for the Mac and I know iCloud will offload to remove the old files from my computer but Mac will still show the file name followed by a cloud icon so you know it’s in the cloud.
In my case, the files are not shown in Finder. Clearly something deleted it off but what?

They are not in every copy of Time Machine. I had to go back a few months. But, Personal and Kids directory have different months where the files reappeared in TM and that show that whatever delete them did not do it at the same time. Strange.

I wish iCloud has a log or something!

1 Like

I suspect Apple’s iCloud not doing this perfectly.

Hopefully your TimeMachine files are ok. And they might be on Apple’s iCloud server (see link above). Dunno.

Have you got any automation running anywhere on your computer?

And still some people wonder why I do not trust cloud services.

3 Likes

It’s your data so it’s your choice what services you do or don’t use. :+1:


I’ve never had a problem with Google Cloud or AWS, and Apple uses both for cloud storage. So, IMO, the problem with iCloud is Apple. They have their own, AFAIK, undocumented way of handling files, directories, and syncing.

I hadn’t used Numbers, Pages, or Keynote in a long time and a few days ago I installed Numbers on my iPad to open some CSV files. When I closed the first one Numbers saved it to iCloud. But I had deleted the unused Numbers, Pages, etc. folders from iCloud a few years ago and my file was not visible on iCloud on my Mac. Long story short I found my file in a hidden Numbers folder located at

/Users/wayne/Library/Mobile\ Documents/com~apple~Numbers/Documents/Transactions.numbers

It became visible when I copied another file into the hidden Numbers folder using Terminal.

I don’t rely on iCloud for much because I can’t trust what I don’t understand.

3 Likes

I have Hazel but it doesn’t touch my iCloud Documents folder.
It only touches Downloads and Inbox folder.
I can’t 100% confirm if there’s no other thing running on the Mac mini (my main) or the MacBook Air. But then, looking at which files are missing, it seems to me like it’s pretty random. Today I found out that an old Project from work had a few files missing but not the rest (again, fortunately for Time Machine, I found them…)

So now, moving forward, I don’t think I want to trust iCloud anymore. I’d rather keep stuff locally. But I can’t copy everything from Documents to a local drive as I am not sure what is missing. Maybe I have to scour the Time Machine backup one by one and grab them. What a mess!

I’d be pissed if it’s ICloud doing but I am not blaming Apple yet as it could be my own fault - some automation I am not aware of. Like I mused earlier, it would be good if iCloud provide a log so we all know what’s happening!

1 Like

You can use the command line to query a log, but I’m not sure if it’ll be useful to you — either because of the historical specificity you’ll need to find the right events, or because it won’t be high fidelity enough to tell you why a deletion occurred.

The tool is called bird IIRC.

1 Like

This may sound strange, but are the files deleted or are they hidden?

I ask because I’ve had a similar issue occur (both with iCloud Drive and Google Drive, annoyingly) where under certain circumstances a file put into their folders is marked by the cloud daemon as hidden. This is freaky because, of course, it looks like it’s deleted things when in actual fact the files are still there, it’s just made them invisible.

I’m afraid I haven’t gotten to the bottom of it yet on my side, and it happens only intermittently—usually with things that go into my iCloud Downloads folder, which Google Drive is mirroring (I think that’s why it happens with Google Drive as well, I don’t think Drive itself is implicated in the problem).

You can check by using the keyboard shortcut Shift + Cmd + . (period) in the relevant Finder window to toggle hidden file visiblity on/off.

Now as for why this happens, I have no idea. The fact that I’ve had it happen with two different cloud services makes me think the bug may be on Apple’s side with the File Provider framework.

And of course yours may be an entirely different problem where it is in fact removing the files, but I figured I’d ask in case it’s actually similar to the issue I’ve been troubleshooting!

1 Like

bird is the daemon (non-user-facing) process (note it’s named bird). The user-facing process is brctl.

1 Like

That’s interesting. I’m not at my Max to check but, when I restored these files from Time Machine, it did not ask me to Replace. So, I’m quite sure they are not hidden.

So I take it that you all didn’t face this and so it must be my current set up. I think I better spend the weekend investigating the Mac Mini, MacBook Air and my iPhone.

I have experienced this exact behavior with iCloud Drive as well – recently too in the last year or two. I also found my missing files in Time Machine. I also pay for the 2TB plan, and have plenty of free space.

1 Like

Oh wow, thanks for confirming that I’m not mad.
I found a folder with photos and inside this folder has 2 subfolders, also photos from two people’s camera. One sub folders is blank. Missing a lot of photos, which I recovered from TM. So, it’s really strange that it seemed random. My faith in iCloud Drive is now at zero! Possibly the only real use is for the Apple Photos app.

I use Photosync to handle automatic and on demand sync of photos from my phone to a folder on my mac. From there I import into Lightroom and move the originals to our NAS server. It has worked pretty much flawlessly for years. Only issue is occasionally it loses the sync on return to location function. No clue why, fix is sync manually then it’s fixed again.

I use iCloud to sync text messages so I can respond to many of them on my mac not just on my phone, also Safari bookmarks although I am now using Firefox on my Mac, wallet for 1 gift card that I have which I’ll delete once it;s used up, weather, shortcuts, actions and actions for Obsidian, Shellfish, Toolbox Pro and Good notes (which I hardly ever use now and am considering removing entirely)

A few years ago I lost some files in iCloud. I spent some time on the phone with Apple support. I believe at the time the files were supposed to be backed up online for 30 days or something like that. Finally, the support person admitted that we weren’t getting the files back. Since then I’ve used Dropbox for almost everything (Documents, etc.), and I’ve never had another problem.

2 Likes

Ugh, I really hate hearing this. I recently upgraded to the 2TB plan and was planning on using iCloud Drive for my cloud storage. Now I’m having serious second thoughts…

1 Like

I have been planning to move to using iOS devices only. That is, stop using my Mac and do all my (limited) computing on an iPad.

My needs are quite simple, media consumption, mail and messaging, document creation and storage (including bills, financial statements, etc.) and most importantly our family photos collection (which is stored in iCloud folders NOT the Photos app and iCloud storage).

I need to be able to trust iCloud!

What do other iOS users do for backups?

Regards,
Peter

This might be worth a new thread of its own.

  • any evidence that iOS backups to iCloud not working? (I ask as I realise the technology in use backups probably different than file syncing which is a complicate thing and there is evidence of unreliability for complex files)
  • does iOS backups to iCloud actually backup up “everything” for an iOS-exclusive user, or only what Apple decides to backup?

I backup iPhone/iPad to iCloud and my iMac, but I realise @25ThirtySix wants to be iPad exclusive.