If I turn the gas on but the pilot light isn't on does the gas keep running?

On a stove it will. On a modern, code-compliant water heater, space heater or furnace it will not, because these modern appliances have electric valves that will not open if the pilot flame goes out. Once the pilot is out, the only way to get gas to flow is to push the pilot button in on the gas valve and relight the pilot flame according to the instructions.

You can check for gas leaking from the pilot burner or main burner by smelling for the scent of gas, as natural gas and LPG (propane) have an odorant added to allow you to detect their presence. (Do NOT use a flame or spark to test for the presence of gas!)