My daughters biological father was not in her life (by choice) until she was 3 years old. He took me to court to get joint custody and they gave him every other weekend and he was supposed to start paying child support. He has not paid any child support and is over $11,000 behind. He started getting visitation the first weekend in September 2011 he got her the first, third and fifth weekends then did not contact us until Dec 28, 2011 for Christmas visitation. Can I file abandonment charges on him and get his rights taken away?

NO, first in general in most states abandonment is not something you file, it is a reason you take him to court to have custody changed.

In Georgia for example, merely not paying support and not visiting is not a reason to take away any rights.

You most likely may get his level of visit changed, and you can try to get him held in contempt for not paying.

You also can get child support enforcement involved. Also ask for a garnishment of his wages.

First, ANY question on law needs to include your general locale as laws vary by area.

Second, Visitation and child support are generally kept separate. So not paying support will not affect visitation and vice versa.

Third, you will not get his rights terminated. But you may get the visitation order modified.

Finally, as Chuck noted you don'r "file for abandonment". Nor, in this case, does abandonment qualify as grounds. But the lack of exercising his the visitation allowed by the courts may be enough to get that visitation curtailed.