Why are aphids mostly found in the stems instead of leaf litter?

That their food sources are around the stems is a reason why aphids tend not to be found in leaf litter.

Specifically, an aphid (Aphidoidea family) is an insect with piercing mouthparts. It pursues a diet which is rich in plant sap. It will be more interested in the fluids which are available in the shoots than the nutrients which are released through decomposition in the debris of fallen foliage around plants.