We have been hearing noise in our house coming from the walls in the downstairs bathroom. There is no rhyme or reason to the timing, except that the rumbling happens more at night. We had a plumber come out for $55 and the guy did absolutely nothing. He couldn't hear the noise so he left. It is not correlated with water running and lasts 3-5 seconds. I read something about a toilet flapper and how to test... is that what this is?

Some plumber, didn't even look in your toilet tank?
The flapper is the rubber thing at the very bottom that seals the water in after the tank fills up again. It gets old and brittle. It's easy to go buy a $5 replacement and attach it yourself, with just some needlenose pliers to crimp the tiny loop. Turn off the valve under the toilet and flush it to empty it while working on it. Take the old one to the store with you. Beware of some that say 'universal' when they aren't, especially for Kohler toilets.
If that fixes it, call up the plumber and tell him his reputation is no good in your neighborhood.

Ok, so even if the toilet ism't being flushed I can have this problem? I've also read about water hammer and air in the pipes. Is it more likely the flapper?

Water hammer makes very loud banging noises - that is air in the pipes.
It's more likely the flapper.
(Possibly some other part of the tank, such as the filler pipe or the overflow pipe or both, which are right at the top so you can see what they are doing, as they try to adjust the level of water. In fact it could be all three making noise, but the flapper is the cause.)
You can't usually see the fact that the flapper isn't sealed tightly.
Take the cover off the tank and leave it off for a day or two and watch and listen!

Thanks!