Why does the sun look little oval at the horizon?

Refraction. It's basically the same reason a pencil put in a glass of water appears to be "bent".