In your original post you said you wanted some good advice. You also said he helped you get over your past relationship as he helped move you out. No, he did not help you get over your past relationship, he replaced the person you were with. You are still as helpless as you were when you were in the bad relationship(certainly not saying this one is good).
You wanted some good advice without attacking you. You have gotten some good advice and reasons not to continue as you are. You have heard from those who have been in your situation and those who have been in the wife's. You still feel you are right. I see that you are saying you are looking at the whole picture but you are definitely not seeing what you are doing to his wife. You skirt that issue. You touch base with it as you do not want to hurt her. Do you not get it? You are hurting her each time you are with her husband. He should be spending that time with her not you. You think what she does not know does not hurt her? That is so NOT true. She is hurt because her husband is not home with her and she is sitting at home alone, lonely want someone to share her time with, someone to hold her. She is sitting waiting because she believes whatever lie he has given her to be with you. You are robbing her of the person she chose to live with. She will find out and the devastation she will feel will be incredible. Her heart will hurt, she will cry like you have many times. She will feel used, betrayed, stupid, abused(Yes abused) heartsick, she will feel her life is over. She will feel she has wasted so much time and wonder how many times he lied to her as she sat home waiting. She will say"What a fool I have been." She will hate you. If there are children involved they will hate you. They will hate their father. They will feel betrayed. THen... depending on the outcome. He will go to her, lie some more and they will get back together. He will hate you. OR she willnot take him back, he will be with you, resentment all around the two of you and he will cheat on you.
Now, I have not attacked you. I have tried to reason with you in other posts. This is reality. You do not want to look at reality. Remember there is the Golden Rule.. Do unto others as you would have them do unto you. Never, do you steal anything or anyone that it will not be taken from you. Anything gotten at anothers expense will not be blessed.
From here, you make some decisions. Your life is a lie. He lies to her, the two of you lie about sleeping together, you know you will again. You lie to yourself saying it is OK. NO do not say you know it is not OK. If you felt that in your heart, deep in your heart, you would not be there. If you want to live in God's light, you have to be willing to live by All of his rules, not just a few select ones.