If a bee flies up to you REALLY close what should I do I know I shouldn't swat at them?

I turn my face in another direction and start walking slowly away. If the bee comes around I turn away each time. If the bee gets in my face, its usually just a carpenter bee, which likes to act aggressive when you're near its territory, but doesn't sting.

You're right not to ever swat at them. That is just about the stupidest thing to do IMHO.