If I leave it off for a while it will build back up but after a couple minutes, go way down again. I am not sure what kind of well system we have. I know it goes through a water softener in the tank before coming into the house.

What you are describing is usually associated with low water in the well. It would be nice to know what type of pump you have. Can you see the pump or is it in the well?