It is attached with ceramic tile along side. I thought about a chisel and hammer. But my wife said I would break the adjacent tile .
How would you do it?

If u have a multi tool, u can cut the holder into sections and pry out. Be sure to wear safety goggles.

Chuck

Yes I do have a multi tool. And a ceramic drill bit Thanks

They make a carbide saw for a multi tool. Cut around the grout and try to pry out the old pieces. BEWARE you may pry out some of the drywall too and then have to put in a patch.

Thanks gang. I use the multi tool around the grout first. Then with a hammer and chisel
I tapped it off.

There you go!