My house is heated by a heat pump. In cold months are bill is almost $500 and we have to wear sweat shirts. I am considering to bring a gas to the house and replace the furnace for a gas furnace. Whole project is going to cost about $10,000. Is it a good investment? Should I keep the heat pump and have so called hybrid heating/ dual heating.
Another question: Should I get a high efficiency furnace? Do these machines have any down-sides?

Its hard to answer your question not knowing where you live. Heat pumps work fine when they are sized and installed properly. When the outside air temperature gets in the 20's they have to run longer to heat your house and become less effective. Have you had your heat pump checked? Low refrigerant charge, defective defrost cycles, faulty reversing valve are just a few things that will cause it to not heat properly. You'll have to compare gas and electric rates for your area before deciding which way to go.

Gas heat will feel warmer since the discharge air is a hotter temperature.
Investment? You would have to do a gas/electric cost analysis to know for sure about any $$ savings.
Dual fuel is what you are talking about with a HP and a gas furnace.
It can deliver a fuel cost savings depending upon the fuel cost analysis.

High efficiency all the way except for the variable speed blower. These have DC motors and controls and cost a fortune to repair after the warranty is gone. I would stick with the standard 3 speed blower along with a 90/95 percent furnace. It will also save you a few$$ when purchasing new.

All new high efficiency purchasing has a down side. They contain more parts and are more difficult to work on hense they cost more over time to own. They do save energy which means you are not paying such a large utility bill.

This is the most important thing out of all of this.
MAKE SURE you are getting a qualified contractor to do the job. The install is actually more important then the name brand of the unit. I have seen a lot of new actually turn to junk because the installer did not know what they were doing.