You didn't answer my question - why didn't you go to the last meeting with the mediator?
No, who pays for the mediation doesn't matter. If you have no income, yes, it could raise the question of who pays to raise the child. On the other hand, if you collect unemployment or some other benefits, it doesn't matter.
Why you don't work does matter.