How are acidic hydrogens different from non-acidic hydrogens?

I'm not sure where the other answer came from, but based on what I learned in my chemistry class, it is dependent on what atom the hydrogen is bonded to. The more electronegative the other atom, the more the electron density is pulled away from the hydrogen, making it behave more and more like an ionic bond instead of a covalent bond. This is referred to as a polar bond. Other factors also come into play, including the size and structure of the rest of the molecule, but the electronegativity of the other atom is the primary factor in whether a hydrogen can be acidic. The polar bond allows for the hydrogen (often referred to simply as a proton, since it won't have an electron with it) to be donated to another molecule that would result in a more favorable, stronger bond. This is the nature of an acid-base reaction.

An example can be seen in Acetate (CH3COOH) which has three hydrogen-carbon bonds, which are non-polar and non-acidic, as opposed to the single hydrogen-oxygen bond on the other end, which is very polar and also acidic.

Note, that there are specific molecules that dissolve completely in water, and where the hydrogen behaves pretty much exactly like an ion: the strong acids. There are seven of these (HCl and HNO3 for example) and this is largely due to the solubility of the conjugate base (what's left after you take off the hydrogen).