How do I get rid of it?

You would have to somehow desensitize yourself to sex or sexual acts.

You could try masturbating several times earlier the same day you plan on having sex. The second ejaculation is always harder than the first and so on…

Also this might not be a (Kosher) answer but alcohol will always desensitize you. I have found just the right amount but not too much will keep you going a long time.