I am in NYC, and I want to apply for sole custody of my child. My child's father doesn't want to be involved. He is paying child support and providing medical insurance. But that is the extent of it. Can he give up (legally) his parental rights, but still have to pay child support?

You appear to have sole custody, sole custody has nothing to do with his rights, It means you have physcial and legal custody of the child. You appear to have that, assuming it is though a court order issued.

Next in most locations, paying child support is considered "contact" even if he does not actually visit.

Now you can't take his rights away unless you have a new partner to adopt. If your new husband would adopt, then his child support also stops.