Joe was just a terrible father, he beat his children, cheated on his wife, took away Michael's childhood, and countless other things. Nothing can change that. He was a crap father.
I believe he has changed for the better and its nice to see that he's patched things up with his children. :)