Why is the penny upside down on one side?

It's a matter of choice which way the front and back of a coin are oriented. Some countries have both sides oriented the same way, and others have them opposite. Some countries (Switzerland, for ex.) have even used both.

Note that every circulation U.S. coin has its 2 sides pointing in opposite directions. You can of course verify this by checking your pocket change.