I agree with what has been posted here but I have to add that your children are forever yours. Don't stay with a man who is unable to love them like you are no matter how much you believe you love him.
You might want to get him back now but what about the future? People don't change that easily on things that are big issues such as kids. He will always neglect your kids and try to push them away. Please don't allow your kids to grow up in a non-loving environment.