Why did southerner states believe they had the right to secede from the union?

They claimed that the USA had started as a voluntary federation of states, and that any of them could leave when they chose.