I have a 1099 employee who was strickly commission based... he offered to take over and manage my compnay when a key employee left. He set up all payroll accounts with an outside service for W2 employees but kept himself on a 1099, wrote his own pay checks and did so for a few years. Now that I need to lay him off he has threatened to turn me in for improperly paying him. Can he do that even through I didn't have anything to do with payroll.

Of course he can. Since you as owner of the company allowed it to happen.
But now you have to, since he has tried black mail he has to go no matter what, and needs to go Today, call him up, change the passwords and he needs to be fired not laid off at this point.

About the worst that will happen, you will have to pay the taxes that you were suppose to pay anyway.

It will hurt him worst, he will have to refile for all for those years and will not get to use most likely all of the deductions he had. Most likely he will owe 1000's in taxes if he turns you in.
But you need to be proactive, report it yourself. Have your payroll company reissue out corrected W2 forms, pay the taxes and get it over with.