Remodeling a house where the previous tenents complained about being shocked while taking a shower. Where do I start?

Look for ungrounded devices, wires across plumbing pipes, loose ground or neutral connections in the breaker box.

This is not a backfeed, and it is typically not something for a DIY to troubleshoot.
Please call in a qualified electrician.

Need to see that voltage with a meter. Try an Extension Cord, plugged into an outlet that has a known good Ground, use this ground for reference and look for voltage with the other probe.
Once found, Turn breakers off one by one to determine which circuit.
If ground are not intact throughout that circuit and a short to ground occurs, anything attached to that ground would become Live, and not trip the Breaker.
Problems at the service can cause voltage through your Metal water pipe.
Seen a pinched Phone line on a Bath tub to give a shock when the phone rang.
It is Dangerous and needs to be dealt with.
Good Luck

I agree with Stan, this is not a backfeed but something that that needs to be found and corrected... Unless you have a professional background please call someone who does.