How small does a horse have to be considered a pony?

no, they have diffrent conformation that's why they are called miniature HORSES

I read your question and i got kind of confused. Do you mean horses? Well, if you do, then horses and ponies may look the same, but there is a small difference that no one barely pays attention too. And that is size. Horses are measured in hands, 4 inches per hand. A pony is 14.2 hands and under. A horse is 14.3 hands and taller.