My daughters father & I share joint custody and I have physical custody. In march of 2011 he moved out of state and has had NO contact with my daughter. No support either. In 2007 he gave his visitation to his mother but he still shares joint custody. His mother never takes her when she is supposed to or even answers my calls. Can I do anything to get custody? Is this considered abandonment? What am I supposed to file? BTW its NYS

I suggest you read this: https://www.askmehelpdesk.com/family-law/child-abandonment-read-first-364259.html

One cannot "give" visitation to someone else. Visitation is ordered by a court.

What custody do you want? Do you want to stop your m-i-l from seeing her grandchild? You already have primary physical custody, is there more you want? I'm just trying to understand what you want so we know how to answer you.

Yes, he may have told you that his mother will be doing his visits, but a child is not a dog where you "give it" to anyone.

He can not give his visits to anyone, and unless you both went to court and altered the visit order his mom has no visits. The mom could perhaps using grandparents rights, get their own visits, but they can't use the sons visits.

And at this point, your ex is not around, and is not using any of his custody. Unless it went though court the mother has no legal rights, but only gets to see the child because you allow it.

You can go back to court to ask the court to give you full custody and not allow him any visits at this time, But he is not visiting anyway ?

He had it legalized that he relinquished his visitation to his mother. He has done this althrough her life and it breaks her heart, an in turn that's breaks mine. I don't want to keep her from seeing her grandmother BUT if she tells me that she doesn't want to see her grandmother then what can I do... I want all rights to her. I think that would be best because in the situation that her father is in, is not fit for a child. So much so one already was removed.

First, who is telling you he had it "legalized". As noted only a court can order visitation so if this was "legalized" he should be able to produce a court order.

Again, if you want to modify custody, you need to submit a petition to the court with grounds for doing so.