How does a bee keeper maintain his honey bee population at a size that will keep his business viable?

This is a two-part answer to your question. In the UK and Northern Europe, most beekeepers are hobbyists so maintaining a high bee population isn't an issue because most of them only want some honey for themselves with maybe a small surplus that they can sell to cover the cost of their hobby.

In the US the situation is very different. Bee farmers hire out their beehives to fruit growers for pollination purposes and the fee that the fruit farmer pays to the bee farmer is his business income. The bee farmer maintains (or increases) his bee population by creating an artificial swarm ie taking a colony of honey bees and splitting it in two thereby creating two colonies where originally there was only one. This is the way that bees reproduce anyway - the bee farmer is just encouraging them to reproduce in his own apiary rather than let them do it naturally because letting the bees do it their own way will mean losing half of his bees rather than doubling their numbers.