We raised and adopted our grandchildren. We have allowed the other grandparents to see them off and on.. the grandfather became confrontational and the boy is scared to go and the girl doesn't want to either. They let the biological mother see them even though we have said no the do it quietly. They want grandparents rights and the kids have been in counseling from the confrontation and do not want to go

Grandparents rights are sometimes given IF it does not interfere with whoever has custody.
This is how I understand how it works.

First, ANY question on law needs to include your general locale as laws vary by area. Some areas do have laws granting grandparents rights under certain circumstances. Others not.

But as the legal parents, without a court order to the contrary, you have a right to control access to your legal child. You need to document what has happened so, if they do choose to take you to court you can provide evidence of why they should not have visitation, or, at least limit it to supervised visits.

We are in Indiana and they have taken us to court. We however do not feel it is in the best interest of the child

We are in Indiana and they have taken us to court. We however do not feel it is in the best interest of the child

Once this case comes before a court, it will be up to the court to decide. Do you have an attorney? If they do, you need one. You also need to document every instance of problems with the other grandparents violating your rules and the children's reactions.

This site has good info including a link to the Indiana statute.
Grandparents Rights in Indiana - Grandparents Rights by State (http://grandparents.about.com/od/grandparentsrights/qt/Grandparent_Rights_in_Indiana.htm)