My ex-husband and I each took one of our 2 boys as our dependent as part of our divorce agreement, although physically we were awarded joint custody. Can I still claim head of household for the child I claim as my dependent?

Yes, both can be Head of Household if you meet the requirements.
Read about the requirements: Your U.S. Tax Return: Filing Status: Head of Household (http://taxipay.blogspot.com/2008/05/head-of-household.html)