There was this movie about a fisherman or a man who lived on a boat and his wife. They hired someone to help on the boat but then the mans wife started to have an affair on the boat with the man they hired. The woman's husband found out and he left.

Could be Young Adam (http://www.imdb.com/title/tt0289635/).