States that did not secede the US during the civil war?

Kentucky, Missouri, Maryland, Delaware.

It was not just their status prior to the war that bothered Lincoln. They were in danger of seceding if they saw the South winning battles, and a Confederate government was actually installed in Kentucky by Braxton Bragg, though it collapsed when he took his army back to Tennessee.