? You don't know, did the boyfriend have it on when he pulled out ? Either he did or did not ?

If you can not "find it" or remove it in one piece, you need to go to the doctor to have it removed. You need to be sure it all comes out. And of course will need to be concerned of pregnancy and STD since when it came off, it failed

He did... but then we switched positions and noticed the condom was no where to be found...

As Fr_Chuck said, it needs to be removed. It can only go so far... it may help if you squat down slightly to try and find it yourself, or you may need your partner's assistance.

It would be wise to consider seeing your doctor about taking an emergency contraceptive, such as plan b, to lessen the possibility of an unplanned pregnancy. This would need to be done tomorrow.