What can cause a starter to keeep turning after engine starts already changed solenoid?

There's a special spring-loaded cog at the end of the starter that's meant to engage when the starter is going and to retract when it isn't. If this special(Bendix) cog don't retract, the starter will remain engaged. You'll have to pull the starter and look at the axle at the end by the flywheel.