Is it okay for me to be upset with my husband when his ex wife wants to hangs out with us and the kids?

What is the woman's reason for wanting to do that?
You are his current wife and if you don't want to hang out with the ex, you should not have to. What is your husband's reason for not seeing this as a problem?

Yes, and its okay to tell him so. She can hang out with the kids when she has her visitation time.

Is she hanging out with the children the same time because maybe an occasion such as birthday of one of their children or holiday. Maybe they both want to be their for the child and when it occurs it is during the time of your husbands visitation. Have they been divorced long? You married long? Wondering if it was fairly new it might be an adjustment for all.

I can see where that could be uncomfortable for you. Has your husband made any comment on it? Does he have an issue with it as well? Just wondering if they had spoken together and come to a decision on their children that maybe you weren't told fully about or maybe you were and you don't agree with it. Yes, you have the right to feel upset. That said if you are the step mom and married a man with children then, you have to realize that she is going to be involved in your lives for as long as you both are married.

Maybe not what you want to hear but think about it this way if those were your children wouldn't you want to spend time with them on maybe holiday or special occasion, even if it was with the new wife. I am sure she doesn't feel comfortable either. You need to try to get along for the sake of the children. If you have an issue with it you need to speak to your husband. Explain your feeling and why you feel the way you do. But you have to remember if you married a man with children, you should have discussed this ahead of time.

The ex wife wanting to hang out with you guys is different from coming together for an event.
She does not need to be hanging out with you. What is your husband saying about this?

I am a step parent myself and I would feel totally uncomfortable with that. She needs to get her own life and you guys need to have yours. Keep you head up and stay strong!