What two things do the peasantry think the term house of usher refer to?

In Edgar Allan Poe's "The Fall of the House of Usher," the peasantry associate the term "House of Usher" primarily with the physical mansion itself, which is steeped in decay and gloom, symbolizing the decline of the Usher family. Additionally, they believe it refers to the Usher lineage, viewing it as a cursed bloodline marked by madness and tragedy, reflecting their superstitions and the family's dark reputation in the community.