I think you have ants in the wall. You may say I'm crazy but once,in another house, I had this faint smell from time to time. I traced it to near the French doors. Getting down on my hands and knees I could smell it on the carpet and saw an ant go through a tiny hole. I went down into the basement and in that spot pulled away the insulation from above the plate. Whew it was alive with ants.
Last night I was woken by a very strong skunky odor. No smell outside. It is ants in the wall of the house. I am absolutely convinced. It happens from time to time in the spring. I have checked outside and there is no evidence of ants entering the house. We are on a slab so they could be getting inside from under the slab. I am not sure there is anything we could do except drill holes in the wall. I just hope they are not carpenter ants.