Does hydrogen peroxide blow up in water?

No. Hydrogen peroxide is usually dissolved in water to make it more stable. The peroxide you buy at the pharmacy is 3% hydrogen peroxide and 97% water.

Pure hydrogen peroxide, which is a dangerously strong oxidizer, can explosively decompose into water and oxygen.