I am not taking his side whatsoever, but maybe there are things that are going on that he doesn't know how to deal with. Again, not making excuses for him but he could be carrying excess baggage from his divorce. You never know what is behind someone's behavior. These things definitely need to be dealt with early. I would take FR Chuck's advice and see if he would go to counseling. You don't want your 1 yr old to suffer from this.
I do know sometimes when I get overwhelmed with things and I start yelling at the kids for things and sometimes stupid things, my wife has to take mae aside and ask me what is wrong. She calls me on it, and I have to decide how I am going to make it right. I hope things work out.
Oh yeah, one other thing I learned through a marriage conference I went to, never talk about divorce even if you think it could lead to it.