I have a game feeder powered by a 6 volt DC battery.

I would like to run the feeder using household 110V AC since batteries keep failing in the cold weather.

Specs on the 6 volt battery are 6V5AH/20HR.

Cycle use 7.2-7.5 V, Standby use: 6.75-6.9V.
Initial current: less than 1.5 amps.

I would like to just plug it into outlet and let it go all winter.

Is this at all possible?

I think you mean 120 Volt AC to 6 volt DC. Search on google and you will find many possibilities.
Also, maybe consider this:
Amazon.com: 6 Volt Solar Panel Deer Feeder Battery Charger Outdoor for Personal Electronic Devices: Everything Else