Why the airplane's left side and right side called as port and star board?

I would assume that the first aircraft actually in use by the military were the dirigibles known as "airships." That and the branch that was most interested in airships were the naval powers more then ground forces. The Navies of the world were very interested in balloons and dirigibles (blimps, rigid and non-rigid) for observation purposes. It being a ship of the air combined being used heavily by naval groups contributed to the same nomenclature as sea going vessels.

That and aircraft operate in a 3 dimensional space, being able to point in any direction with only one consistency, up and down. So left and right can be in any direction, and since an airplane can be in any position in the air, how does one decide what is left and right when an aircraft is upside down?

Update from a 747 pilot: The left and right side of aircraft are indeed referred to as port and starboard. Have you ever noticed the lights on the wingtips of an aircraft? The left side has a red light and the right side has a green light. Go look at a boat sometime... you'll notice they have the same lighting pattern. There are many other similarities between airliners and sailing ships (i.e. the aircraft commander is referred to as a "Captain" and the pilots wear naval style uniforms). As to why aircraft are referred to as sailing vessels is a completely different question to be answered elsewhere.