Why is absolute value is use to express the distance between two points?

A distance is a measurement, and does not change based on which location is the starting point. (Unlike physics where the direction is considered)