It has a remote control

Mine does too. I leave the actual switch in the "on" position so I can use the remote only. Mine does it because someone close to me also has a remote on that same signal, so when they turn theirs on ours comes on too. You could try leaving the switch in the off position until you're ready to use the remote

Remote employ radio waves to control the unit and there are only so many frequencies available that, at times, someone else may have the same frequency. Good news is you can change your frequency by changing the arrangement of the dip switches in the hand held and the receiver. If you want further info just let us know but tell us brand and model please.

I've been too lazy to take my fan down and do that. LOL. I keep saying I will... and then I don't.

You do notneed to take the fan down to change the frequency. You open just the canopy and find the receiver dip switches, move all switches to O, or any combination you want other than what they are now. Then open the battery compartment of the hand held and make those switches match what you did in the receiver.