I have an older gas furnace with a belt drive blower motor. The pulley was slipping on the shaft and wore a groove in the flat spot where the set screw sits. We drilled a small indent into the flat spot and changed to a longer set screw and really cranked it tight, but the pulley again loosened up. Any suggestions what is causing this to come loose? It sounds like an air compressor running in my basement

See if you can move the pulley a bit without throwing off the alignment. If that doesn't work, drill and tap a shallow hole the same pitch as the set screw and use red thread locker.