I have depression

Yes, it is self harm. Your body needs food in order to survive.

Have you talked to a doctor about your depression? Are you on any medication for your depression?

There's help available, so why suffer through this? Go to your doctor, tell him/her what's going on. Seek counseling for these issues.

It's interesting how you word your question.
Depression often goes along with lack of appetite. People may eat a lot less and just nibble and not want to cook, so will live on cereal right of the box or cold pizza.
But you say 'refuse to eat,' which has a sound of hunger strike to it.
I apologize if that is not what you mean, and you are just having to deal with people who say your low appetite is self destructive.
However, the fact that you are on the computer asking questions suggests a desire to do something about your depression, and I hope you don't get so low that you don't want help to get out of it.

How much are you "not eating" are you losing weight to a serious level.

I mean I may fast for a day or two, ( some of my friends longer) for religious reasons

And if you are still eating, just not feeling like it,