I think I got a keloid INSIDE my nose from my new piercing but I believe it was caused because I won't stop touching it! I know it's my fault, I'm learning the hard way. Will it go away?

No, it won't go away. But it doesn't cause any harm and its inside your nose.