You don’t want to wait too long - you should always have a few week’s cushion when transferring in case of problems with your old registrar (eg getting the EPP transfer code in time, removing any Registrar lock).
Also, you don’t need to wait; many registrars will spot you up to a year left of carryover time (or at least offer it as a promotion) - I know it’s part of namecheap’s transfer policy. Typically, transfers include a one-year renewal, so you keep the remaining time already registered for that domain and add a year to that.