What is the meaning 'load balancing' in computer science?

Load balancing is a term used in the field of parallel programming, in which multiple CPUs (or even multiple different machines) will be doing work at the same time. Load balancing itself refers to allotting each CPU an amount of work (a 'load') which will ensure that they all finish processing at the same time.

The whole point of parallel processing is to reduce the amount of time it takes to perform a task. Without load balancing techniques, we may have one CPU doing 90% of the work and another doing only 10%. This does not give us a good speedup.

An ideal load balancing algorithm will distribute (100/N)% of the work over N processors.