Switching fire off and never using again. Is this sufficient to prevent carbon monoxide leak when a leak has been detected?

Carbon Monoxide is a product of combustion, so technically, yes... stop the combustion you stop the Carbon Monoxide at the source.

You should however either repair or remove what caused the problem because at some point you might forget about this issue, or worse someone else that never knew of it starts it... and has a defective detector and could die from it.

Yes,, no fire no carbon monoxide, as the burning of gas is where the carbon monoxide comes from

Both answers are great but maybe you should be telling us more. Fire in what. How was the CO detected and attributed to this yet to be named unit. Reason I am asking your CO detector may get set off and you conclude it's the furnace but it could be the water heater, fireplace, etc... Please tell us more