Okay, let me set this straight.
The pill "freezes" your ovaries from releasing an egg while you are ON the pill. The week that you are off you are considered unprotected.
There are hundereds of birth control methods out there and each one is different, just as each one of us women is different. Each pill affects each woman a slightly different way.
In no way can we say for sure that you can have sex during your "off" week and not get pregnant. That is just dangerous advice. Therefore we cannot generalize when giving an answer to a question such as this.
I, for one, was on the pill and took it religiously every day at the exact same time every day, and I have 2 sons currently in the military to prove that the pill is not 100% effective.
Itgirlragdoll, your question was not in the least foolish. What is foolish is the doctor who jokes around instead of answering a very important question such as yours.
You were very right in asking for advice here since your doctor would not give it to you. I hope I have helped answer your question and if you have any more, please do not hesitate to ask.