This whole situation is a bit much. As I see it you may feel more for him than he does for you. You two are Fwb. If it was anymore than that he was ask for it to be.
You two may have known each...