I live in NJ and will be working in Philadelphia. In addition to a state income tax, non-residents who work in the city of Philadelphia have to pay a city wage tax. I am assuming the "income tax" between the states is reciprocal, but do I get any credit from NJ for the additional wage tax.

You are correct - there is a reciprocal agreement between NJ and PA, but the Philly tax is a "wage tax" is not part of the agreement. When you file your NJ tax return you can get a credit for the Philly wage tax as taxes paid to other jurisdictions by filing schedule A with your NJ-1040. See the following web site:
Income Tax - Credit for Taxes Paid to Other Jurisdictions - NJ Taxation (http://www.state.nj.us/treasury/taxation/njit14.shtml)
And page 9 of:
http://www.state.nj.us/treasury/taxation/pdf/pubs/tgi-ee/git3w.pdf