Why is my toilet backing up in the tub and not flushing?

There is a blockage of some type in the line going out.

Chuck's correct. You have a blockage downstream from the tub and the back pressure is interfering with the flush action. The drain has to be snaked and cleared. This can be done by snaking from the lavatory roof vent or pulling the toilet and going down the closet bend. If you come down from the rood vent put out enough cable to reach the base and 20 feet more. Good luck, Tom