Do I need permission to use the name of a college in a nonfiction book?

No.

If it has some sort of intellectional property right to the name (copyright, trade-name, etc.) your use would still be under "fair use" exception.

Just don't write anything derogatory and untruthful about it.

Agree, if you say bad things about it, have a dozen murders happen in the dorm, they may look at a law suit then.