My boyfriend and I are having an argument whether the phrase goes, "In a matter of speaking" or "In a manner of speaking". Can someone help us solve this war??

I've always heard and said, "in a manner of speaking." Manner in this case means way, a way of speaking. Matter doesn''t work in this case although it could be used if there was an understanding that manner was what was meant.