My air conditioner is on a 25 amp breaker and should be on a 20 amp max (per manufacturing). All I need to do is take out the old one and install the new one- right. Do I need to worry about the size of wire when installing a lower breaker or just when increasing it?

Reducing the rating of a circuit breaker is never a problem, the assumption is the wire is larger than needed. Just to be sure, check that your wire is a minimum of #12, it may be now or may be #10, if on a 25 amp CB.

Someone may have incorrectly installed 25 Amp Breaker on a pair of 12's, because the breaker kept tripping. The breaker protects the wire. On long runs, you may need to go several sizes larger, to reduce voltabe drop.