Remove old thermostat and replace with honeywell model RTH3100C the old one has terminal O/B
NEW one has this terminals are separated need to connect wire to O or B FOR HEAT PUMP

PLEASE HELP

I believe you will need to find out weather the reversing valve is normally energized in heating mode in the outside unit OR if the reversing valve is not energized in the heating mode.

Your old thermostat was probably set to either O or B in the programing OR sometimes there is a slide switch that would select O or B on the thermostat itself. You might look for that and if you find it then the selection will be easy. Now I will say that most heat pumps energize the O in cooling so that will probably be the one you are looking for.

Your instructions

http://customer.honeywell.com/techlit/pdf/PackedLit/69-1889EF.pdf