I did a photo shoot for a student claiming he is going to a photography and art school. He said he needed 5 subjects to take photos and that it would be a nude shoot. There were no written documents, but the verbal agreement was he would take no face shots and they would not be published online. Now he saying he has a face shot and will post the pic if I don't do certain things for him. I was paid 20 dollars, so is this considered blackmail still? If I have messages from him saying he doesn't deny what he said about the agreement, can I charge him of anything? Who should I go to?

... I was paid 20 dollars, so is this considered blackmail still? ...

Probably. Report him to the authorities for extortion (or blackmail). The problem in prosecuting him would be proving that he demanded that you do "certain things" in exchange for honoring his verbal agreement.

You where paid for the photos, but he should have gotten a written consent, but you agree you where paid for the photos.

He may post them and use them as he wishes, since you agree he paid you for the use of the photo. After he does, you could sue him for perhaps a percent of the profit, if he makes money off them, since there is no written contract to profit of photos.

This is blackmail, the minute he says, he will post them if you don't do something.

If you were willing to take nude photos, you had to know there is always a chance they will go public, sometimes in mistake, but he shows them to school, others gets copies, School may post them on a school site.

Sorry, tell him, post them you are proud of them, or report him to the police for blackmail.
If what he is wanting is sex or some form of sex, there may also be other laws being broken
"attempted rape" since he is trying to force sex though blackmail.

Report it to the police, they can help you get proof of the blackmail. Then they can prosecute.