A couple things.
1 - Alois, his father never abandoned his family, he however died when Hitler was 13.
2 - he lived in Vienna with illusions of grandeur, thinking he was going to be this great artist and when it didn't work out, he lived poorly, throwing his money away on Operas and selling...