Can I pass a hair drug test if I'm not a frequent drung user?

It depends on when the last time was that you did drugs, and what drugs you did.

I think it stays in your hair for 28 days

Drugs, most of them stay in the hair follicle for up to 90 days. Yup that's 3 months.

28 days is a hair shy of 3 months (sorry for the pun). Again, it all depends on what drugs you are talking about and when the last time you used.