When did you go is a sentence fragment?

Yes, "When did you go?" is a sentence fragment because it is missing a subject. Adding a subject, such as "Where did you go," would make it a complete sentence.