If my car is uninsured and is parked and empty and an insured driver hits it isn't he responsible to pay for the damages whether I am insured or not?

Yes. If your car was legally parked your insurance status does not matter. It's the other driver's fault. Put in a claim to their insurance carrier.