The exhaust fan in my bathroom is extremely loud and is getting on my nerves. Is there anything I can do, short of replacing it, to quell the noise?

Not sure what would quiet one down, I will say many manufactureres has the sound level down to .3 sones.
You can't hear them. Ducting can affect sound, they use Larger Duct.
Here's one:
http://www.ventingdirect.com/panasonic-fv-05vk1-whisper-ceiling-50-cfm-exhaust-fan-3-sones-4-duct/p531499