What holds all the instructions the computer needs to start up?

it is the ROM . The ROM has firmware which controls the booting up of the computer during start up. The booting software, known as BiOS (Basic Input Output System) give instructions to the POST (Power On Self-Test) which then runs tests to on the computer to make sure that all the devices are well connected to the computer. Sometimes this process is so fast that when observed, it appears as if there is are no tests being run by the POST process.