I've been friends with a guy for a year and he's dating my best friend.
We hang out all the time together. When it's the three of us, we have fun
But if it's him and I hanging out, he shuts down. When his girlfriend is mad
At him he will text me like crazy but then he'd barely talk to me. It
Pisses me off because I don't know if he cares about me as a friend
Or not. I am so confused, am I not a good enough friend for him

Answers yes. He is off limits to because he dated your best friend.

Has nothing to do with you, just that he has to keep a distance, so his girlfriend doesn't get all jealous, and make life hard on him. It happens all the time no matter how close friends are, or have been.

Romance just makes people possessive sometimes, and its hard when there are conflicts, and sides to take. Its probably best to back away from them both, and have other friends besides just them, to hang with, because then you won't take his behavior so personally. There are boundaries between a guy, and his girlfriends friends even though you are all friends, just because she has to get a lot more of his attention, and time than he gives you. Just to avoid conflicts.

How old are all of you?

He's 26 she's 19 and I'm 21

You don't need to be friends with him. Why would he have a relationship with you apart from his girl friend? Why do you want there to be?
He is staying within established boundaries and you should probably do the same.
Stop hanging out with them so much.

I barely ever hang out with him alone and plus he doesn't want to stop being my friend. He and my best friend are helping me through the loss of a
Loved one. I told him I can't be friends with him but he wants us to be
Friends. He said" just cause she and I are dating, doesn't
mean I'm gonna not want to be your friend."

When his girlfriend is mad
at him he will text me like crazy but then he'd barely talk to me. It
pisses me off because I don't know if he cares about me as a friend
or not. I am so confused, am I not a good enough friend for him

I'm not getting what you see your problem is? You say he barely talks to you then you say he wants to be friends.
Maybe he is secretly attracted to you in which case you need to leave him alone.
If you tell him you can't be friends with him, it does not matter what he says. Stay away from him.