Can I take them to court and sue for harassment

Did you ever talk to him "over the fence" and try nicely to ask why do you and I have a problem? Swoller your pride. You need to know what his problem is with you and try to compromise before calling in the lawyers. Call him up and ask for a meeting.
He is angry about something. Or crazy and unreasonable. It is up to you to try to resolve this amicably. A good fistfight always made good friends when we were young. Now days with 911 the winner is 911 with your fine money.

Sueing him will be a long term costly love affair. Your lawyer will love you to broke-ville and three years later you may get to trial.

Spend all you can on day and night video cameras and voice recorders to document the harassment.