Double drive use for iCloud problem

I have a very strange problem on one of my Macs (but not the other two): The space on disk used by files in iCloud seems to be double what it should be. My iCloud drive used space is about 4.5GB which is correctly recported on two of my Macs. but repored (in Finder->Get Info as 4.49GB, 4,491,565,974 bytes (9.7 GB on disk))

Going into termimal and looking at individual files gives similar results; ls -l reports about half the file size as du -h does for space consumed for a file.

Any ideas?

Does this Mac have a larger drive than the other two?
Unless things changed, drive space is allocated in blocks, with larger drives having larger blocks.
Lots of small files exacerbates the problem.

2 Likes

I just checked on two of my systems, and they both allocate 4K blocks. Creating a single character file shows up as 3 bytes in ls -l and 4.0K in du -h, so there is some discrepency. However that would never exceed the size of a single block allocation. Also, this is only showing up in iCloud synced files. Purely local files show correct useage.

More curious: This seems to be limited to files that have synced in. If I create a file in the iCloud synced directory tree on the affected computer, it shows correct usage using du and correct size using ls -l

I was having difficulty with this computer not syncing iCloud files earlier this week, so I turned off iCloud drive, re-installed MacOS, and then turned iCloud drive back on.

I agree that this will be it. Your Hard Drive is storing 4.49gb of data, but due to it having files which are smaller than the Hard Drive block size, it means that it’s using 9.7gb of space.

In your example, your 3 byte file is using 4kb of storage. A 5kb file would use 8kb of storage. It’s just the way drives work.

The choice of block size when formatting the drive or accepting the default for your drive size will set the block size, there are trade offs here though, smaller block sizes create more management overhead, larger block sizes lead to more wasted space.

The only other thing I can think of is to check for snapshots that might be holding onto space.

tmutil listlocalsnapshots 

Info on mounting, deleting, etc. snapshots:

2 Likes

I appreciate your thinking about this, and I’m now arguing only because it’s fun to hash out technical details and learn new things:

I don’t think that this is how file systems work in general. Or yes it is how they work, but in that example, a 6, 7, or 8KB file would also use 8KB of space. For “large” files, unless APFS is designed in such a way that it regularly uses only half of a block allocation for data storage, the difference between the number of bytes stored in a file and the number of bytes used on disk should only vary by a small amount, ideally less than the size of one block.

If it were an issue with inefficient block use then I would expect this to be a problem everywhere on the computer, but it’s not. It’s an issue only with files that are synced from iCloud drive. I would also expect it to be an issue on the other two computers (which are also using 4KB blocks), but it’s not.

Creating a file on the errant computer’s iCloud directory tree, or copying a file from its local filesystem into the iCloud directory tree results in the reported file size and disk use for the file being roughly the same, as expected.

Further, copying a file from an unaffected computer via a different means (not syncing it though iCloud Drive, but using a flash drive, also results in expected (by me) rough correlation between reported file size and disk use.

Finally, only one of three computers is exhibiting this behaviour. It’s very suspicious to me that disk use is showing almost exactly double what it should (again, only for the iCloud Drive synced directory tree, and only for files synced from, but not created locally). It’s almost as if something is being counted twice when being synced.

As I’m typing this, it occurs to me that I haven’t done a free space comparison before and after sync…

Duplicate files are handled differently in APFS and iCloud copies may not be considered dups.

1 Like

Which part of my tiresomely long screed are you referencing here? :slight_smile: (I don’t think there’s any problem with duplicate files, only a discrepancy between file size(s) and disk use.)

APFS shares disk space for duplicate files. If the iCloud files aren’t recognized as dups for some reason, wouldn’t that increase your disc usage?

Edit: I’m just spitballing here. :slightly_smiling_face:

IF you are using APFS, you can just stopp locking onto those informations about the file/folder size, because they were not right in almost all cases, due to APFS. The Information there within the Finder is a kind of a relic to the “older” file systems in use, where there where more reliability at this informations, and more problems using those systems with SSD´s.

So the only way to really learn about the disk space in use remains with Disk Utility.

And, in your specific case, if those Files-Sizes are doubled, I would also place a bet on a snapshot for TM, or an other Back-Up-App with a similar technique.

What if you copy a file from the iCloud dir to another (non-iCloud) dir? Is the file size trimmed?

If it is a APFS, a duplicate would have the effect the other way around, so a higher file size in the first numbers, and a lower at the “(XX GB on disk)” part, as the dups are only duplicated within the File-Address-System (I don’t remember the right Engl. Term at the moment) of the drive, but are “physically” only one time stored on the drive.

No one else seems to know for sure so I’m just taking wild guesses. :slightly_smiling_face:

1 Like

One of my Drill-Instructors always said:
“A strong guess is always better, than a weak proof…”

2 Likes

I think the feature that you’re talking about is copy-on-write, where only alterned blocks of a duplicated file are actually separately stored from the original file. In APFS if you “duplicate” a file it will only use a tiny amount of extra space until you begin to alter it in some way.

1 Like

I’ll try that when I get home.

Each Drive has a kind of a Header, where all files are listed by there names, and location on the drive.
And if you duplicate a file within APFS, there will only be a second entry there for the duplicate, but with reference to the “old” location of the file.

One other thing that I’ll do is use Disk Utility to check free space before and after I sync my iCloud drive.

Have you tried any of the utilities that claim to free up purgeable disk space?

For my part, I would not. Since Apple began using purgeable disk space and letting multiple volumes share the same container, I have given up trying to understand my disk storage beyond a fairly shallow depth of understanding. :slightly_smiling_face:

I haven’t tried any of those, no. I’m not usually a fan of those kinds of things.

I have a couple of things that I’m going to try this evening and I’ll report back here with my findings :slight_smile:

1 Like