I live in New Jersey and work in New York city. I know that I'm subject to New York STATE income tax. As a ny non-resident, am I also subject to New York CITY income tax?

No - the NYC tax is for residents only. Since you live in NJ and work in NYC you are subject to NY State and NJ taxes, but not NY City resident tax.

If you are present in a state, then any income earned during this period, must be reported to the state. So you must file NY state return.

To NJ you must report your worldwide income for the year. Since you paid taxes to NY, you will claim credit for those taxes on the NJ tax return.