What is the part of the computer that contains the hard drive?

The hard drive is usually installed inside the computer case. Some refer to this as "the box" or "the system unit."

Some even refer to this part of the computer incorrectly as the CPU. That is due to misunderstandings and in some cases, carryovers from the old days. The central processing unit was not a single chip like it is now, but a collection of many chips and transistors which spanned multiple circuit boards. So in the early days, all those components collectively made up the CPU. Now, the CPU is a single chip, and most of the other components are included on the motherboard, and they all make up the system unit which is together inside the case.