I agree you can't unask them now but when the trip is over, make clear to your kids that they have to ask you in the future before they invite other people to join the group, and remind them when you next make plans.