Why don't you get out of breath so quickly when you do physical activity?

It should never be difficult to breathe when doing cardiovascular activities. You should be breathing deeper and more rapidly. If that is the reason, it's because your heart requires more oxygen to unlock the potential energy in your body. If by difficult you mean it hurts, you should consult a medical professional.