I'm on H1B, married (spouse doesn't work on H4), 3 children, income below the w4 mentioned rate for married filing jointly so can apply for tax credit. The total allowances work out as Married / 11.

Does this sound right?

Use the worksheet on W-4 and work out your allowances. You can file joint return as residents and claim your children.