A few weeks ago, I came home to find my doorbell button stuck in, and the transformer was very hot. There was also a buzzing noise coming from the chime box. I unstuck the button and eventually the transformer cooled back down.

Now, the button is lit up, but when you press it, the chime box just makes the buzzing noise (as long as you hold down the button for). Do you guys think I need to replace the transformer or the chime box? Thanks!

Measure voltage from transformer, if low, possibly internally shorted windings, If OK, maybe the chime coil has some shorted turns?
If you only are using front door, try switching front door wire to back door wire, that coil may still be good.

I switched to the back door wire and it works now. I'll use it like this until that coil goes out (hopefully doesn't happen again), then replace the chime unit. Thank you for your help!

Since the problem was caused by the stuck doorbell button, I'd change the button as well. No point in letting it happen again.