When the Civil War began in 1861 how many blacks known as Freemen or Freedmen were living in the US?

Oh, dude, like, I'm not a history buff, but I'm pretty sure there were around 4 million Freedmen living in the US when the Civil War kicked off in 1861. Can you imagine being a Freedman back then? Talk about a wild time.