What doesn't the town like about dolphus Raymond?

The town disapproves of him living with the black people of the town. Dolphus has a black wife and mixed children which is unheard of in Maycomb. However, due to the fact that Dolphus has money and land and has a "drinking" problem the town puts up with him.