My brother's ex girlfriend has decided that she would no longer like to be a part of their daughter's life. She wants no rights to the child and no obligations to her. My brother and the mother of his 2 year old child are both 18. After the mother told him she doesn't want their baby anymore, he decided it would be in the best interest of the child to live with our older brother and his wife in a different state while he finishes college and builds a life his baby girl deserves. All parties are in a agreement to this arrangement. However, this is not to be an adoption. The child's mother would like to give up all parental rights, the father would like to give up guardianship. Are the courts going to allow the 18 year old girl to give up rights to her child if it is in her best interest and her father does not object?

This explains it all - https://www.askmehelpdesk.com/family-law/signing-over-rights-read-first-116098.html.

This has to be presented to an Attorney where the parties currently reside due to the complications of a second State.

I've read this and several articles similar to it. I am worried a judge will deny the request. Is there an alternative if the judge denies her giving her up her rights?

No, not at this time. If the father marries and the wife wishes to adopt, yes, then the father could go back in front of the Court, the mother already wants "out" so she'll sign and the father and wife adopt.

The fact that she can't legally give up her rights does NOT mean she can't surrender custody.