I've noticed these bumps that are the color of the rest of the skin on my lower inner labia. The occur on both sides. Any chance that this could be normal? They don't bother me at all.

How old are you?

If they're not causing pain nor looking inflamed then they're probably normal. It would be something you should show your doctor/OB on your next check up.

@nazia. OP stated these bumps do not bother her in any way. If it were pubic lice, there would be irritation and itching.