He feels that you've given away a part of you that once belonged to him. Whether it's his right doesn't matter, this is what he believes. I've been there, and if I had to do it over again, I would've ended it at the first shred of doubt instead of prolonging it for a year and a half.
Sorry, but there really isn't anything you can do to convince him otherwise, this relationship has run it's course.