I have been talking to Apple support over the problem of slow Time Machine backups, which first appeared with Catalina, and has continued into Big Sur.
I was trying to backup a new M1 MacBook Air with about 350GB of data on the SSD. The backup disk was an external drive on a Mac Mini.
Like you, I thought that the backup had frozen, but if I let it continue, it did complete - although it took a long time: just under 30 hours! Subsequent backups were taking maye 7 or 8 hours - and of course, on a laptop this means that backups simply were not completing.
The first thing that Apple told me is is that I should make sure the backup disk is formatted in APFS, Having done this (and I had some initial trouble getting Time Machine to recognise the disk, but resolved that eventually by rebooting the laptop!) the time for the first backup dropped significantly - to about 11 hours, and subsequent backups now take about 2-3 hours. Using a wired connection to the Mac Mini did reduce the backup times, but not by very much. By contrast, using a directly connected drive the first backup takes about 3 hours, and subsequent backups a few minutes.
I am still not happy with backup times, but using APFS for the backup drive are certainly significantly improves things compared to using traditional MacOS formatting. So far as I am concerned this is still an open and on-going support situation.