How can I tell them to get out of my relationship with out me making them feel bad.
They are worried about you, and your child, and if your staying with them, you have to give consideration to the rules of their house. They don't want him around, and that's understandable and something, given the circumstances.
I wouldn't want to be in that situation either, so you and hubby better get your heads together, and get some better meeting arrangements, and visitation limits. That's how you get them out of your business.
What you think your going to bring your own chaos, and problems back home? Be glad you have a place to go while you work things out with your husband, but also be mindful of their wishes also.
Is there something your not telling us? Why would you hit your husband any way? His reaction wasn't right, but neither was yours. What's to work out, unless there are other problems beside anger and violence, which by itself is enough.
Its hard to take your side when you admit to starting it. Maybe you should go home, and work this out. Then nobody is in your business but your husband.