Did the north have less manpower than the south?

Not at all, it was the other way around. In the 1860 census the northern states had 21 million inhabitants and the southern states 9 million. But the southern total included 3 million slaves and a half million free blacks. So there were 21 million mostly white northerners and 5.5 million white southerners, meaning the north had a four to one advantage in manpower over the south. No one can give a precise figure of the number of men who served in either the northern or southern armies during the war but the north's men in uniform outnumbered the south's two or three to one. The north had the more difficult task, militarily though. It had to invade and conquer a vast area to win; the southerners had only to defend and not lose, to perhaps eventually gain their objectives. So the north's manpower advantage was just sufficient to achieve the conquest of the south. Virtually every man in the south of military age was in the Confederate service; the north never really came close to completely mobilizing all its manpower resources.