If you've seen the classic film, Kane is a very egomaniacal and possessive man.
Therefore the quote is totally fitting with his character, who sees everything in terms of himself---a narcissist.
I realize that this is an old post, but it still strikes me interesting. I think that it really goes a bit further than just relating to Kane. Everyone wants to have love be on their own terms.