I live in Ontario, Canada and I'm wondering if using a spy pen camera to record someone is against the law? Could I take legal action against this individual who recorded me without my knowledge?

Mostly it depends on the purity of the purpose. Lets say in a case of home protection. Or to see if someone is stealing from you at your home. Those are different from outright spying. If someone is overtly doing it for unreasonable issues then it most likely is illegal and an invasion of privacy. Was this outside of the home ?

Mostly it depends on the purity of the purpose. Lets say in a case of home protection. Or to see if someone is stealing from you at your home. Those are different from outright spying. If someone is overtly doing it for unreasonable issues then it most likely is illegal and an invasion of privacy. Was this outside of the home ?
Yes this was outside any home. It was in public and in restaurant. I want to know if I would have a case against this individual?

Was it someone you know? Or was it a stranger? Did you see what was filmed? Was it focused on places it shouldn't have been? All of that comes into play.

I think the main issue is what the video is used for.

Its not illegal to film people in a public place.

I think the main issue is what the video is used for.

Its not illegal to film people in a public place.

I agree. There are legal lines and definitions as to where and when also including purpose of matters like this one.

Yes I know I was being recorded after the spy pen was pointed at me. I asked the person to erase the video but he refused.

Does this mean anyone can go take a video of a person in public place and keep the video even if he asks to remove the video and asks not to be filmed?

The essence of it is that even though a "spy cam" was used since it wasn't in a covert manner and it was in a public place it is rude at best but it doesn't sound illegal. It would be different if we were talking covert type of activity.