What determines a hard wood from soft wood?

The idea of 'hardwood'/'softwood' is a little confusing. Since many 'hardwoods' are softer than some 'softwoods'! What's the difference? Hardwoods are any deciduous tree that has a non-scaly or needle like leaf. Softwoods are generally, non-deciduos ('evergreen') trees with a needle like or scaly leaf. Think of Maple vs. White Pine.