When the furnace kicks in and is running I can now hear a knock every 4 or 5 seconds that appears to be coming from inside the furnace. What could it be? When it stops running the knock stops. It doesn't happen when the blower is on by itself.

Usually a sound of this nature is the blower motor with loose bearings or fan wheel. Sorry to say but since you say that is not the problem you will probably have to call for a professional HVAC service person to take a look.