Why are Lennie and George forced to live a nomadic life?

Of Mice and Men takes place during the Great Depression. Since work was scarce, they had to move about to find it. However, they had to move more often than one would suspect because Lennie gets in trouble by scaring women. When he touches a woman's red dress in Weed, she runs away screaming, which implies that she was afraid he was going to rape her.