Sound to be like he's testing the waters before he dives in. It seems to me that he's afraid of getting hurt and therefore wants to know how you feel before he admits how he feels. Follow you gut and heart, make a list of pros and cons, be honest with yourself and him.
Number one thing though, if you don't think you'll be happy with him or in Japan then you shouldn't go, this decision requires allot more conversation between the two of you, obviously there isn't much of that at this time. Talk, talk, talk, list, list, list, and then decide.
Good Luck.
