When you are next alone like in one of your bedrooms (not during sex), say jokingly: "so hey are we seeing other people or just each other?" And go on from there in a light hearted manner. If he wants to be exclusive you'll know by the end of this conversation. Importantly before you start, you need to know what you want, like I know you want to be exclusive, but what do you want to do if he doesn't want this? You need to decide what you will do beforehand and implement it.

Like if you know you can't be in a relationship with him that isn't one on one only, you would (still being light hearted) then end it but nicely. And still be friends and stuff, without the sex stuff.

If you know what you want then you will be a lot happier.