If that's the case, then wouldn't we have a lot of breakups? Why date at all then? If the most beautiful/hottest/prettiest/handsomest man or woman walked into the room, there's virtually no way we can't think about it even for a second. Unless we're robotic?
As long as we don't act on those feelings. I think that our level of self-control against these sort of feelings is part how much we like/love our significant other.