Dad left when I was 12 and he always took his anger out on my brother who is the middle child.

My brother is now 17 and I'm 19, I still am angry at dad so I hardly talk to him now but my brother moved in with him and has been living with him for two weeks now. Early hours this morning my brother called me upset... He said he told dad that he was never there for him when he was younger and my dad responded saying "well when you were first born I didnt even know if you were my kid" and he still thinks it to this day. My dad told me that a few months ago and I went off at him. Now my brother wants to come home and I don't know how to comfort him or what to say to our dad. HELP!