Does acceleration always mean that an object is speeding up?

No. As used in physics, the Word "acceleration" means that THE VELOCITY CHANGES. More precisely, it refers to how quickly the velocity changes, in symbols, dv/dt.