There is about 4 feet of water pipe from where the pipe enters my basement to the shut-off valve. During serious overnight freeze, I shut off the water and bleed the pipes before I go the bed. But there is still water in that 4 ft pipe before the shut-off valve. If that is exposed to freezing temperatures, can't that freeze and possibly burst?

But there is still water in that 4 ft pipe before the shut-off valve. If that is exposed to freezing temperatures, can't that freeze and possibly burst?

Yes if the basement temperature drops below freezing. Easiest solution to this problem would be to wrap that section of pipe with thermostatically controlled heat tape.

Simply use pipe heating cable and you never get like this problems with frozen pipes.

Like this one http://www.pexmall.com/30-ft-pipe-freeze-prevent-cable

Are you saying the air temp in your basement gets below freezing? Where are you located?