Like we have said you need to get out of the house and have a place to stay established before you can get a restraining order. You can't get a restraining order and then still be stuck having to live with him so your mother needs to talk to the police, a lawyer and the court about her best plan of action FIRST. She needs to file for sole custody.
It all rests in if there is a court order issued and finding the best way to go about things.
IF he does have court orders for your custody then it could be a bit harder.