First, this isn't a matter of "luck", its a matter of law and a judge's determination of what will be best for the child.
What has been left out here is why they have to take you to court over this issue? Have they asked for time to spend with your son and you have refused? Have you been willing to have them visit but not unsupervised and/or not overnight?
California (see here:
parentsrights.com - California State Code Grandparents Rights) does recognize grandparent rights of visitation under certain circumstances. So to prevent they getting rights you will need to show that its not in the best interests of your son. I would strongly suggest getting an attorney to help you with any hearings.