What role did the theory of the states rights play in the outbreak of the American civil war?

First, States Rights is not a theory, it is within the Constitution. Since the Constitution primarily limited Federal Government powers, all things not specifically granted to Congress fell into the purview of the States. The South also argued that the United States had been formed as a voluntary federation, and that any state was free to leave the USA at any time. The Federal Government at the time had made several laws and tariffs that seemed to serve only to the Northern States advantage. This, with the South's feeling that the Federal Government was over-reaching its legal powers, led to the South deciding that the Government was no longer valid. They formed their own country and their own Constitution based heavily on the US Constitution, though it had more constraints on Federal law.

The same concerns about Federal over-reach are in the news currently.