I live in NY but work in NJ. Do I have to fill up a W4 for NJ or for NY ? Do I owe taxes at both states ? Please suggest how it works. I moved in from Chicago app 2 months back

You will have to file taxes in all three states.

1. You will file NJ tax return as non-resident and report all the income you earned while in NJ.
2. If you have moved out of Chicago and it is not temporary or transitory, then you will file IL tax return as part-year resident reporting income while in Chicago.
3. You will also file NY tax return reporting income you earned while in NY.
Read: Your U.S. Tax Return: Working or Living in Two or More states

To clarify, you will pay taxes on your NJ-sourced income to BOTH NJ (because you earned the income in NJ) and NY (because that is where you live).

NY will give you a credit for the taxes you pay to NJ to avoid double taxation.