Melissa,
Could it be an inner ear infection? That is a possibility as well.
Brain tumors do not always have pain as a symptom. In my Great Aunt's cat, we used prednisone which could have reduced any swelling produced by a brain tumor (which is the symptomatic treatment of choice for cancer/tumors) and in fact would have reduced any swelling caused by multiple issues. We still do not know what caused Tigger's problem but it seems to be under control for now.
In your cat's case, it could very well be the toxoplasmosis.
Click Here for an intersting article on Toxoplasmosis.
I hope, for his sake, it is. Just be careful handling any feces. It takes 24 hours for toxo to sporonate so cleaning the litter box 2 or 3 times daily reduces the risk to you of contracting toxo greatly. Of course, it's generally not an issue unless you are pregnant.
I suggest, if your cat isn't eating well, getting some oral feeding syringes and force feeding some meat baby food (such as veal, turkey, chicken, etc) or canned cat food such as Eukanuba's Maximum Calorie. Max Cal has a pudding-like texture which makes it easy to syringe feed plus it has extra nutritional support for cats who are ill.
Regards,