My son is five years old and I split from his father four years ago.
We were sharing custody but after his father refused to bring him home, there was a few months where he did not see his son as I asked for him to sign a bit if paper saying that he shared time with our son. After seeking legal advice I was told without this I would have a hard time getting my son back if he decided to take him.
My ex refused to sign anything and left it 6 months before he decided to take me to court.
I won residency of my son without even applying as the court seem to distrust my ex as much as I do. Plus shouting at the judge and my solicitor didn't do him any favours.

My son has been told by my ex not to talk about my boyfriend and when he has his father has told him 'i don't want to hear about it'.
My son asked me to speak to his dad as this was really upsetting him.
My son never knew what he could or couldn't say and withdrew into himself at his dads.
His dads response was to yell and moan at our son asking him constantly what is wrong and getting cross when he didn't answer.
Our son is so scared of upsetting his dad as he can be aggressive in his attitude.

My son see's his dad every other weekend and for half the holidays by court order.
My son came home yesterday and said he had a worry (which is our way of him telling me problems and then we blow them away).
He said on the way home daddy had told him the reason he didn't see him for a while was mummys fault and he had to speak to a special man who could tell him when he can see him.
I explained its no ones fault and he shouldn't worry about these things as mummy and daddy both love him and will always be in his life and that's all that matters.
My sons response was to ask me if it was his fault.
This was heartbreaking and of course I told him its never his fault.
He often says daddy says nasty things about me and my ex has sent me texts saying he will tell our son how nasty I am.
I have never bad mouthed his father and refuse to as I know this would upset him.

I really don't know what to do for the best my son is now not sleeping and its causing him a lot of stress at 5 years old'
I have tried talking to his dad but he is aggressive and refuses to listen and says if he wants to be childish its his choice.

How can I protect my son from all this hurt his dad is causing him?

First, you made a mistake when you first split in not getting the courts to order custody and support.

Second, you should explain to your son, that when two people split up, it can be upsetting to hear about the other's new partners so he shouldn't talk about that with his father.

As far as his father bad mouthing, I'm not sure what UK law (I assume you are in the UK because you referred to a solicitor ANY question on law needs to include your general locale as laws vary by area. ) is on parental alienation. But its something you should explore with your solicitor.

Thank you for your advice.
I am from England in the UK.
We had hoped to keep the relationship civil enough to not have to go through the courts as we agreed on most things at first and the court said there was not enough we disagreed on to need an order when we first split.
My son does understand this and does not go out of his way to talk about my boyfriend but when his dad asks him what he has been doing and who with my son should not have to think about his answers at five years old.
My son is constantly questioned about my boyfriend but also told not to talk about I'm and is very confused. My Ex has agreed himself he was in the wrong in doing this and has since said he will tell our son it is okay to talk about anyone but I fear it is too late for that as my son is still worried his dad will get cross with him if he mentions my boyfriends name even if by accident.

With the distress and sleepless nights it is causing my son I think I will definitley have to take this up with my solicitor.

Thank you

Given the above, I would have the son see a counselor. A professional may be able to help him deal with this and help you and the father deal with your child.