I had an old Coleman Evcon furnace, the problem is when the flame came out doesn't come strong because the fleame stay right at the nozzle of valve pipe,

Sounds like the gas valve needs replacing or the water coulmn pressure needs adjustment per manufacture specs.