When switch comes out of set position, the pilot light goes out.

Most likely the thermocouple needs to be replaced.