Moving a Photo Library from one machine to another

I got my new M1 Mac mini and copied the Photo Library (563 Gb) from my notebook over to the new system hoping to save some sync time. Then I turned on iCloud/photo sharing and ended up with a photo library that was over 900Gb.

Is there a right way to do this or should I just turn on iCloud Photo sharing and let it re-download all of the photos.

Did you select the new library as the system default?

If so, it might be macOS is just checking the consistency, but not actually re-downloading everything?