Is there anything that kills thistles that doesn't harm the hay around them?

2, 4 D will kill the thistles and not harm grasses. It will kill alfalfa and other broadleaf plants. There has been concern that the manufactiring process leads to the production of dioxin as a byproduct. This is the chemical in Agent Orange. It is serious stuff and I would think twice before using this pesticide. If you do decide to use it, follow all safety precautions to the letter.