I am indian citizen and a non-resident in US, and have been asked to fill in the W8Ben by my bank on the interest bearing account. I am not sure, whether I need to 9a on the W8Ben?
Also, I have another question, Do I have to fill in a W8Ben form for only the interest bearing accounts or all accounts (like regular checking )?

Yes, you DO need to complete Line 9a to claim the treaty article that exempts you interest from U.S. income taxes.

One Form W-8BEN for each bank should be all that is required.