What side did the border state fight on?

They supplied men to both sides, and fought within their own states amongst each other (see: Bleeding Kansas).