I have

Ask him if you can come along sometime. If nothing is going on, he shouldn't have a problem with it.

I would just show up and say you missed him and wanted to see him.

But of course I most likely would just tell him I am not happy with him doing this any longer and after work his place was coming home.

Follow your instinct.he should consider your feelings first and if you are not comfortable with this he should change his behavior.This:we are just friends"is a huge red flag and the fact that he is secretive about it.If things were harmless,why exclude you?Go with him next time.Don't ask,just do it.