The issue is, you have not been with him much, You know him from his best behavior on short visits, not every day life. It will take a lot of work and I would suggest that you understand the problems that will happen after you are together, every day, every night all week, after week.