No you can not use a notarized letter. It must be a court order sorry. But what you can do is a mutually submitted court document that you would both sign, file with the court and you would not need a court date or anything. You would just file it and the judge will sign it then it will become and order of the court. To be more specific regarding court forms you may be able to do this with I would need to know what state and county you are in.
::Edit:: I can not take credit for finding this but here it is thanks to Lady B:
Passport Requirements for Minors Under Age 14 - Associated Content
For minor children traveling outside of the United States, effective in 2007, a United States passport is required for re-entry into the United States. The key factor to passport acquisition is the necessity that all minors, under age 14, appear in person at the passport location and, even more difficult, the passport application must have the signature of both parents or legal guardians or the custodial parent must provide adequate documentation securing the parent's authority to make sole travel decisions on behalf of the minor child.
For single parents who do not have contact with the other parent, and are unable to obtain form DS-3053, the passport can be issued to the minor child providing the custodial parent has a legal document which supports the sole custody rights and, in addition, the court custody documents must not indicate the minor child is restricted from travel.
And you may be able to find some helpful information on this web site as well:
International Child Abductions: A Manual for Parents - Consular Affairs