Was Oil of vitriol once used to treat canker sores in the mouth?

Highly unlikely. Oil of Vitriol is an old name for Sulfuric Acid (H2SO4). This acid is highly corrosive and can be quite dangerous. It's dangerous to work with in any available concentration, and not used directly on patients in any first aid context.