Somebody tried using duct tape to stop the stovepipe from "puffing" at the seams and at the brick wall. Now we want to remove it, but the tape leaves sticky adhesive...

At the library, I found Goo-Gone to be fantastic at removing adhesive residue off nearly anything. It should be available at big-box stores, hardware stores, etc.

Get a small can of mineral spirits or lacquer thinner.