What is the difference between plain and ugly?

When referring to a person, plain means rather ordinary features.

Ugly refers to a variety of factors perceived as unattractive to a certain viewer. The concept of ugly varies quite widely among cultures and social classes. The single factor that sociologists could identify when large numbers of people were quizzed on which photographs were of ugly people was that feature imbalance on the face was often identified as ugly.