You know the BLUE cap that's on the end of the applicator? Well I forgot to remove it.. So is it true what goes up must come down??

No, not at all... many people have things such as tampons or condoms lodged "up there" and need to be removed by a doctor to prevent infection.

You can try to remove it yourself by squatting down with your legs apart but being a hard object I would recommend a trip to the doctor.

Never be embarrassed about going to the doctor.

I can feel it but I can't grab it, only moved it. Would a hot bath do any good?

You wouldn't want to do anything to risk it being moved further up. I persoanlly have never heard of doing this so I can't say if it would help or not.

Thank for your time.. I got it out.

Good to hear, and congrats on saving on the doctors bill! :)
Just make sure you check for any pain/discomfort to be sure there was no damage done :)