How to test a 120 to 24 volt transformer can you use ohms if its out of the system?

Use volt meter and test for 120 on primary side and 24 on secondary side

Yes if the transformer is out of the system you can use an ohm meter to check if the transformer is OK.
Another way is to connect one side of the transformer to 120 volt line use a voltage meter connected between the other high voltage line and the second line to the transformer, if the 120 volt side is good you should get a reading of 120 volts. Then if that shows good connect both lines to the 120 volt side and then check to see if you have low voltage on the low voltage side.