I'm not sexually active and it hurts slighty when touched, it has some stuff under the skin I can feel it. Could it be from mastubating and creating friction? Will it go away?

It can be from "rough" masterbating or masterbating too often. It can be very likely a ingrown hair
But could also be several other things, without a proper examination no one can do anything but guess.