The main problem is that the copy in finder never seems to complete, it initially identifies that it has about 450GB to transfer from the 500GB drive. When it gets to “450GB of 450GB copied” both values then start going up. When it got to 1TB of 1TB copied (from a 500GB drive? How’d it do that?) I stopped the copy.
I’ve tried this a number of times, reformatting the drive, changing the permissions and starting the copy, each attempt resulting in the amount of data copied exceeding the size of the drive it’s reading from.
The only thing I can see that I’ve done different to the instructions is that they state the scheme should be “GUID Partition Map” though I don’t see an option to set that. Only an option to set the format for which I selected “Mac OS Extended (Journaled)” as per the instructions. Both old and new drive show as MBR partition map so I figured maybe this wasn’t a problem. I doubt that’s related to the copy issue but mention it for complete-ness
Anyone seen anything like this before and is there a simple fix?
Oh, still on Mojave, haven’t jumped to Catalina yet and the Mac is a 2017 15” MBP if that’s any relevance…
I am not sure if this will help, but there are instructions here about how to change the partition map using Disk Utility in newer versions of macOS. I hope this might be of help to you as I have never tried copying TM backups to new drives. When I’ve replaced drives in the past I have just started Time Machine from scratch (along with a full disk CCC bootable backup).
Hummm, so the partition map doesn’t appear to have been the issue. Same problem as before, just still copying data over when it should have completed the copy.
Zero items left to copy but it’s still copying “something”.
Fiddle Sticks!
Hello @stu_w, I’m sorry that the reformat of the new disk did not fix your problem. As I mentioned in my previous message I have never tried to move TM files from one disk to another so I am out of ideas to help you.
You probably need to exclude your new drive (and the old TM drive) in TimeMachine. TimeMachine is probably beginning to back up the new drive to the old drive, which is being copied to the new drive. Maybe.
Thanks, Good thought @JohnAtl but one of the steps is to turn off Time Machine (Uncheck the option for back up automatically). At this point Time Machine hasn’t been told to use the new drive either, that would be the next step if I could get it to successfully copy over the backup files. Well, it
probably is copying over the files, the issue seems to be it doesn’t know when to stop copying over the files! When it does it’s initial “preparing to copy” it finds over 564,600 files to copy so I’m wondering if just the sheer volume is sending it a bit nuts.
Ah, I see what you mean, I didn’t think TM did back up external disks, interesting. Thanks for the heads up, I’ll have to experiment with that as it might be useful to include certain USB drives etc in a TM backup.
Unfortunately, in this instance I don’t think it’s the cause as TM was switched off at the time while the copy attempts were taking place. I had the preferences window open in the back ground to keep an eye on it and can confirm it wasn’t doing anything either. It’s useful to keep in mind though, thanks!
This looks like it’s more complex than one would think.
When TM does a backup, if a file already exists, it stores a reference to the existing file by using a hard link1. When you try to copy disk-to-disk, these hard links can’t be copied as is, so the files they point too are copied - over and over. That’s how you get over 1T from a 450GB drive.
I think the best solution here is to start a new backup on the new drive. If you can, just put the old drive on the shelf (or in your bank vault) for a while until you’ve built up history on the new drive.
Edit: you might be able to use ChronoSync or some other utility to copy the files over. It looks like ChronoSync can fix the hard links so they will work on the destination drive.
Another option if you’re really adept as using Terminal is to use the rsync command. Fair warning, that’s going to get really hairy.
Still, I think the best option is a new backup.
Thanks, that’s a good link, I’d had a bit of a search but not turned up anything quite so useful.
Started reading the info page for cp, seems there maybe a way to tarball it though I’m not convinced there wouldn’t be permission issues down that path.
I think your suggestion of sticking the current drive in a safe place & start over is probably the best one for preserving my own sanity if nothing else.
I did chuckle at the comment in that thread suggesting it should have been called ‘Time Bomb’ instead of Time Machine.
Thanks @ACautionaryTale, @JohnAtl also mentioned rsync (so that’s two votes in favour ) . It’s not something I’ve used before but I’ll have a read up on it before I decide what to do. There seems a few options, it’s deciding which is the best (safest way) to go. Thanks!
(tl;dr version = I gave up and just started a new backup on the new disk.)
I played around a bit using tar to copy the files, even with sudo I was getting permissions issues reading the source folder. Got round this by adding Terminal.app to “full disk access” on the “privacy” tab of “Security & Privacy” under “System Preferences”.
This appeared to sort that out but then when it was moving files / folders over it was spitting out errors about setting permissions all over the place so I killed that.
Did some reading up on using rsync for this but found a page suggesting that this too had a bundle of problems when it came to hard links.
By this point, it’s sucked up that much time I just wanted to get past it and the easy option was just to start a new back up.
I had success with SuperDuper. This past week I had to move a 2.6TB Time Machine Backups.backupdb to a new volume. I tried several methods:
the Apple recommended Finder method failed,
rsync -aHAXEN /sourcepath/Backups.backupdb /destinationVolume appeared to work but resulted in a much larger data set, and
the good news is that SuperDuper worked without any significant explosion in storage.
This was with the trial version of SuperDuper. My only issue was that I found the SuperDuper UI a bit confusing in that when it completes there is an “OK” button. Unfortunately, on my first attempt I hit that button which then deleted the destination volume and started over. So I wound up having to wait another 24 hours for it to complete the second time. I’m a CCC and ArqBackup person so this was really human error on my part ¯_(ツ)_/¯. Also note CCC can not currently be used to move the Backups.backupdb.
An alternative approach with TimeMachine backups which makes them much easier to move is to use the sparsebundle format which TimeMachine defaults to for network attached drives. Since sparsebundles when unmounted appear as a single file, they are easy to move around including even to the cloud storage of your choice.
A useful resource for understanding Time Machine and the way it works was organized by James Pond prior to his passing away. That information may be found in the Internet Archive Wayback Machine’s www.pondini.org. It is also currently being made available here.
