I am asking because my daughter lives with me and only me since she was a year old. He left years ago due to an argument in which I told him to choose between us or his alcoholism. He chose the alcoholism and left. I have been generous to a point. I let him see her for a month during the summer every year. But this year he asked to have her again. When I said no he said he would call the police. We never had a formal court hearing on custody, but at one point we were supposed to pay me child support. I felt bad for him and cancelled it, because he was struggling. Of course now I'm wishing I hadn't. I just want to know if legally speaking he can do something like that. He is not the legal guardian.

If there is or was a court order to pay support, then somewhere he is listed as the father, and visitation or custody was also described by the court.
What state is this that orders support with neither DNA nor name on birth certificate?
So as far as you have described this, no, you can't deny him, even if you dropped support. Did you do that formally or just let it go?
You need to go back to court.
How old is the child? Won't she miss him? Why do you want to deny him?

.... I let him see her for a month during the summer every year. But this year he asked to have her again. When I said no he said he would call the police. We never had a formal court hearing on custody,
...
but at one point we was supposed to pay me child support. I felt bad for him and cancelled it,
...
. I just want to know if legally speaking he can do something like that. He is not the legal guardian.

Even if he is not on the BC, you have, by allowing a CS order to be entered, acknowedged he is the father. Therefore in most places both of you would be considered to have shared joint custody. In other words, at any time, he could take or keep the child, should he feel inclined to do so. Both of you each has the right to do that.

But he is wrong about the police. Normally the police will not have the desire or the power to intervene in a custody dispute between the parents, absent a specific custody/ visitation order.

You say he is not the "legal guardian". One doesn't normally use that phrase so as to describe parents of a child, but, yes he is. Both of you are.

The DNA test was done in Missouri, but I now live in Georgia. He once resided in Georgia in another city, but he up and moved back to Missouri in the middle of the night. I do not wish to let her go a second time, because it is so far. Because he has no job or reliable transportation I don't trust that he would bring her back. I also fear about the living conditions where he lives. I have been informed of alcohol abuse and illegal drug abuse happening in the home and do not want to allow her to go for fear that she may be taken from both of us. She is 5 years old and has told me that he only drinks beer when he's there. I did not openly ask her this. I asked her if daddy drank a lot of kool-aid and she replied that he only drinks beer. This raised a big red flag for me and I do not feel like it would be a safe environment for her. As far away as it is there is no way that I can just up and drive over there to get her should something happen.

Unless there is a specific visitation order in place, you can refuse visitation. But, he can go to court to get a visitation order. Whether he gets it depends on whether he can prove he is not a danger to the child. The good news is that the court order will mean he has to return her.