My house is about 20 years old. The city water pressure is very high in my area, I find that when I first turn on the water on any inside tap, it's very strong for about 5 seconds, then becomes normal. I've replace the water regulator leading into the house. Could it be the new one is faulty also or is that normal for a high pressure water area?

By "water regulator" are you referring to a PRV,(pressure reducing valve) see image? If so it sounds like the new valve's leaking water into your system and building up the pressure. I would replace the valve. Good luck, Tom,