Are check advance companies legal and what will they do if you don't pay them?

They are legal in some states, like Virginia.

If you don't pay they sue you and get a judgement which will lead to a garnishment.

I am sure there are legal and illegal ones like in any businenss.

And it depends on the terms.
If they merely loan you the money, they will sue you, and get a garnishment,

Many will have you sign a form to allow them to garnish wihtout going though courts.

In others they have you write them a check for repayment, so if it bounces they press criminal charges if you don't pay it