I have a leviton 057 dimmer switch , two black wires and one green. Have 12/3 wire in box which wire goes were?

When you say 12-3 I assume there is a black, red, and white, along with a bare ground.

Really depends on how this 12-3 is connected at the other end. There can be any combination of how someone connected the cable.

What type of switch was connected to the 12-3?

A 3 way? If so, the dimmer you have is not the correct one, you need a 3 way dimmer.

Also, if switch only had 2 wires, remove switch and connect dimmer. Like above, if 3 wires were attached(not counting ground), you would need a three way dimmer.
There are dimmers that use neutral also, not your case.

If you have 3 wires in the box, can you identify the color of the wires? If one of the wires is green, that's the "ground wire" which is to be connected to the green wire in the dimmer switch and to the box, if the box is metal. The two remaining wires are the common or "hot wire" and the "return wire" which should be connected to the dimmer switch.