I have a job in New York City but my office was located in New Jersey and on my paycheck I get taxes taken out for New Jersey as well as New York. I get NJ state income, disability, unemployment and NJ workforce. Along with my NY taxes. My question is, should I pay all of these taxes ?

Do you live in NJ or work in NJ. If not, then your income is not subject to NJ taxes even the employer is in NJ.
Read Your U.S. Tax Return: Working or Living in Two or More states (http://taxipay.blogspot.com/2008/06/working-in-two-or-more-states.html)

It looks to me that you work in NY City, which makes you liable for NY state income tax regardless of where you live.

If you LIVE in NY city, you are also liable for NY City income tax as well.

If you live in New Jersey, you will have to file a NJ state tax return, but the credit that NJ provides for payment of the NY state income taxes will MORE than offset any NJ tax liability, so you will pay NOTHING to NJ. For this reason, have your employer STOP withholding NJ state ncome taxes.

If you do NOT live in NJ, then the NJ disability, unemployment and workforce should NOT apply. You should speak to your employer about getting these withholding stopped as well.