Name the part of the computer that stores all information and software?

The Hard Drive is the main storage unit in a normal, desktop computer. This is the only internal component in a standard computer that has information when the machine is turned off (all your saved files, the operating system (such as Windows), etc).

There are other components (such as RAM, or Random Access Memory) that store data, but this usually serves a special purpose, or is only at run-time.