How should one dispose of old button batteries?

That will depend on where you live. Batteries can be recycled, so a recycling center should accept them. In California, batteries cannot be thrown away in the trash, so most curbside recycling services will accept household batteries.

Talk to your local trash collection service to see what they recommend.