If the father of my baby's wife does nothing but calls me a whore and harlot do I have to let my child be around her?

You have to let the bio father have the baby if there is a court ordered visitation. If they are talking bad about you in front of the child, you take him back to court to ask for a change in visits or prohibit her from being around child,

You can not decide anything, you have to do it though courts.