Well, I'm not the experienced with life and all but here it goes...
I guess him having sex with you while your asleep can be compared to something he's into like a hobby (not exactly the best word I could use). Usually when people have hobbies or do something they like it because there is always something that happens when they do it that gives a familiar feeling inside that feels good, or satisfying, either way doesn't give any type of negative or uncomfortable feeling.
So if he hasn't done this all his life then either there was something he used to do (doesn't necessarily have to be sexual) that gave him same feeling or he used to do something that gave him that feeling but found it more present in what he does now. You can't really just go off and label him as a rapist because you don't now for sure if it's the fact that he's "attacking" you while your defenseless that he likes.