Which is the cheapest horse you can get?

It all depends on the breed and possibly the colouring and personality.

There are some horses going for free because the owners cannot afford them in this economy and no one will buy it. Most of these horses will not be good ones however.