My heat pum starts blowing hot air then it will go cold and reduce the air temperature by 3 degrees. If I turn it off then turn it back on again it will heat right back up. Is this normal?

No not normal.

Unfortunately its hard to diagnose over the internet, you'll probably have to call a tech to look at it. Can you give us more info like how cold is it outside, how warm is it inside, what do you have the thermostat set to, how warm is the air coming out of the vents, how long does it take to drop 3 degrees, etc

It usually happens in the morning when the thermostat is set to go from 66 degrees to 68 degrees. The morning have been running in the mid to upper 30's. I am not sure how warm the air is coming from the vents but it does start warm. I have 2 HVAC systems and they both tend to do this at times. It takes about 10 minutes to drop the 3 degrees when it is blowing out cold air. Thanks for all your imput. Any help is appreciated!

Unfortunately the drawback of heat pump systems is they drop out about the mid to upper 30s. When the temp outside drops to mid 30s, the condensing unit can't absorb anymore heat. Heat pump systems come with an electric heat coil so that when the temp drops to mid 30s, the auxiliary (or emergency) heat kicks in and heats your house. Does your thermostat have auxiliary or emergency heat?

Yes, it does. It is really strange though... it brings the temperature up to the 68 and then it is about an hour later that it happens and I have to shut the system off. Again, this happens with both units. Also, something that happens sometimes the system shuts off and there is a significant hum for a minute or two and then a whoosh and it stops. I don't know what that is either and if I should be concerned...