How can stores who buy jewelry tell if the jewelry was stolen?

Well there's really know way of being 100% sure. The only way to be absolutely sure whether jewelry is stolen or not is to either produce the jewelry yourself, or buy from a reputed, trusted vendor. In addition, if a jewelry piece comes with a letter of authentication or a GIA or AGS approved certificate then odds are the jewelry is not stolen.

If you're interested, here's a great article about the biggest jewelry heists in history. The biggest diamonds stolen never usually show up on the market because they'd be recognized and that would expose the thiefs.