You don't stay with someone just because you don't want to hurt their feelings, and someone who loves you does not treat you with such disrespect.
I don't think you really want to be dating this guy. You keep asking if this was a mistake. I think it was.