How long does it take to die when you inhale helium?

If you just inhaled a bit from a party balloon you would not die, though there have been cases of people becoming unconscious.

People have died from trying to inhale helium directly from a helium gas cylinder. They died because the pressure of the gas punctured their lungs and the surrounding blood vessels, and they drowned in their own blood. CPR is no use in this case. This would have happened with any gas, even a cylinder of oxygen. It was the intense pressure that did the damage, not the gas.

People died who crawled up inside a helium filled balloon. Helium is not poisonous, but they died from lack of oxygen. Air has a normal load of 21% oxygen, which is enough to keep us living. In that case, they were breathing helium with no oxygen, and they would suffer a sudden and immediate lowering of the oxygen levels in their blood. They would lose consciousness as they were trying to grab a second breath. If they continue to breathe the pure helium they would die in a few minutes from hypoxia (lack of oxygen).