How do you get rid of dirt in a swimming pool without a vacuum?

This sounds as thought the laterals in your sand filter are worn. Laterals are horizontal spokes at the bottom of your sand filter with tiny holes in them that let water through but not sand. After a time the holes get enlarged and start allowing sand through and back into the pool