Frankly I would be more disturbed by the fact that this is a co-worker and you haven't reported him for sexual harassment. But that's another issue.

There are many people that have open marriages....