Who worked in a manor?

The peasant and his family, of course.

-----

In most places, for most peasants, the house was occupied by an immediate family.

In many parts of Europe, peasant families were able to have servants living with them. These were usually young people who had moved away from their families and had not yet started their own. Calling them servants might make the families they lived with sound to modern people much more important or wealthy than they were. They could have servants and still be rather poor. Possibly the house also had the chickens in it during the winter.

In many places, the peasant houses were what is called long houses. In these, there were often several peasant families living together, often together with other people who were single and not part of families. The long houses were big enough that very often people lived at one end, animals were kept at the other, and there were large storage areas for grain and other foods. The long houses were most common in northern Germany, Holland, and Britain, though oddly, more among British Celts than Anglo Saxons.

There is a link to a related question on what serfs houses were like below.