I've recently moved to NJ and want to update my w-4 so I am not paying those horrible NY taxes. I understand I still will have to pay NY state, but not the city. Do I update my NY w-4 (certifying I do not live in NYS, NYC, Yonkers) and that's it? Or do I need to fill out a NJ w-4?

How about exemptions? I now have a little baby--if I claim her as an exemption, does that mean my husband cannot?

You need only update the NY state equivalent of the Form W-4.

If you are BOTH working, only ONE of your should claim the child on their W-4. I recommend that the parent who makes the LESSER salary claim the child.