First, he seems like he is being very distant and doesn't want to start a relationship and you are kind of forcing him.

Second, yes you are hoping for too much! You just started talking to him...