... but when I shut off ALL water to the house it stops. Isn't this drain for draining the tank? Shouldn't it leak until the tank is drained??

Not necessarily, apparently the valve only leaks when under pressure. When you turn off the water to the house and open a faucet all the pressure is relieved.

The valve has a hose thread. Get a hose cap and put on the valve.

If it is the drain and not the TPV valve, you can try to tighten it, turning to the right. If it still leaks and has male threads on the outside of the valve, almost all do, go to the plumbing section at HD or Lowe's or ACE etc and get a brass HOSE thread cap with a washer. Screw it on the valve and it should stop. If it is the TPV, DO NOT put anything on the pipe to try and stop the leak, get it checked out and fixed properly. .

Once again, no posts shown and then we add the same answer. Go figure!