If I sign over all parental rights to a child in nj do I still have to pay child support?

Yes otherwise many fathers would be doing that just to get out of child support.

There are MANY threads here and in the Family Law forum that discuss this. I suggest you read some of them.

Family courts are unlikely to allow you to relinquish your rights just to get out of paying child support. Generally a Termination of Parental Rights (TPR) is granted only when there is someone waiting to adopt the child or the parent represents a danger to the child.

As noted first of course most courts will not allow you just to give up your rights, esp if the other parent does not agree. And yes of course you will still have to pay child support, support is not a right but a obligation.
If the other parent has another partner that is going to adopt, then your child support can end.