When were horses first used for transportation?

by the 1920s, horses were a rarity. People did not just stop using them like that. They just gradually died off and the car took over. In answer to your question, I would say around the 1910s.