The water pump to my well doesn't shut off some times. The water pump pressure switch sticks on, either because of the switch or the pressure gauge. If I tap the switch box it will click off.

I have shut off the circuit breaker and loosened the nut (on the advice from a plumber). This helps for a short time. Do I need a new gauge (the current one is rusty and corroded), a new pressure switch, or both?

I amsurprised

I am surprised that your plumber did not recommend replacing the switch, I would at this point.

Agreed. It is a cheap fix ($25.00 or so for the switch) and not difficult to do if you are comfotable with working with electricity. Be sure to also check the small pipe leading to the switch for any sort of obstruction.