You're both adults, if you both consent then there is no problem with it.
As for the age issue, it's only as big a deal as it is in your own head. There are plenty of people attracted to older and younger partners, and many that enjoy very fulfilling relationships in age gap relationships. There are as many benefits to sleeping with a man in his 50s as with a man in his 20s. And there are as many dangers in sleeping with a man in his 20s as there are with a man in his 50s.
HOWEVER - the fact that you couldn't answer this for yourself and had to ask a forum of strangers suggests that you don't know your own feelings. If I were you I would refrain for this reason. If you really want to, then you won't need to ask others if you should. Give yourself a little more time to learn about yourself and your heart. Just because you're 18 now doesn't mean you absolutely have to rush into sex with anyone at all if you don't truly feel comfortable.