I want to stop my sons father seeing him due to the amount of drugs him and his family smoke if there a chance of doing this? I don't feel my son is safe when with them and am not happy to leave him on his own

Is there a custody order in place? Does he have visitation?

If you can prove in court that he is using drugs and has a criminal record of drug use.

Merely "your word" will be considered a bitter ex saying anything they can. So you will need evidence that can be presented in court.