Is it legal to own a used AT 4 rocket launcher?

This weapon is almost certiantly classified as a military weapon where ever you live. Unless your country allows possession of military-grade high explosives, or the like, the answer is no. No country comes to mind that would allow possession of such a weapon.