Our water heater has limescale buildup and makes a very loud rumbling noise when heating. We tried flushing it but that didn't really help.

Would installing a water softener help clear out the lime in the water heater and prevent it from coming back?

A softner will not help. How often do you flush the tank?