So today I turned on my iMac, and smoke started coming out of it so immediately I turned it off, after five minutes I turned it back on and now it is working fine, is this something I should be worried about? What could have caused it and is there a way to prevent?

Sounds like you got a dust bunny and had a burnt sacrifice in power supply. You should have it checked out there is a chance that there is still conbustable matererals there to burn ot that the burn caused some damage.

I agree buy a can of compressed air and clean it out I have and old iMac g3 and had a similar problem .