I'm remembering a TV show that now that I think about it was very very trippy. I used to get the VHS tapes from the public library but I'm very sure it was a TV show. It was a live action show and had a lady that resembled the teacher from Magic School Bus and the kids in the show would come into her colorful crazy trippy house and they would come in from this weird entrance and come down into her house. Very weird.

The only thing I can think of for you is Out Of The Box. I don't think I'm right though

I don't think it's Out of the Box. It was "Mrs. something something" I think...

The only other thing I can think of that fits your description is Hello Mrs. Cherrywinkle
Compare Prices on Junior Collection Hello Mrs. Cherrywinkle I Could Play All Day