Something is offensive when it is intended to demean and belittle. When its based not on any facts, but simply on an unreasoning belief.
Your question does need to be put in the context of why you are asking.
Just because a viewpoint is offensive does not mean the person isn't entitled to that viewpoint. People are entitled to their viewpoints no matter how ridiculous, stupid or unreasoning they are. But then I have the right to view such a person as dumb, hateful and bigoted.
The second part of this is what a person does with their viewpoint as excon pointed out. I believe that a person should be allowed to do what they want up to, but not including when that something infringes on another person's right to do what they want. Clearly the offensive nature of racism interferes with other people's rights so, expressing and acting on those viewpoints becomes offensive and in some cases, illegal.