Can you only use your hands in sign language?

No. A lot of sign language involves gestures and facial expression and using other parts of the body in conjunction with hands.