I have my H1b Visa but I have not entered in US till now. But now I want to file for H4, for my wife so that we both can travel together. SO can I file for H4 without entering into US? Is it possible?

Yes, as a matter of fact, this is what you should do.

She needs to submit DS-156,157 with...
Proof of marriage/relationship
Proof of H-1... copy of your passport (every pages), letter from your employer, approval notice from immigration, etc.

So I have to go with her for H4 stamping?? Means for the interview...

Please clarify if you have the visa issued from AMCON or just the I-797 approval notice from USCIS. If you have the visa issued, you do not need to appear with her for the visa interview. Have her bring everything Lawanwadee mentioned previously, including a recent paystub.