Why do tea bags float on cold water?

Assuming you mean literally float, as opposed to giving the appearance of floating e.g. by using a magic effect such as very thin string, there are a few ways this could be achieved.

First, the teabag could be made to float on an air cushion - i.e. by propelling air upwards beneath the teabag at a constant speed so that the teabag 'sits' upon the air cushion.

Alternatively, you could take the teabag into a weightless environment, either artificially created for example in an aircraft (see e.g. http://en.wikipedia.org/wiki/Weightlessness#Reduced_weight_in_aircraft), or natural (for instance, space).

Possibly the most interesting method would be through diamagnetic levitation. This is a phenomenon in which some substances when in the presence of a magnetic field produce an opposing magnetic field which repels the original. Although if you're using natural magnets this can only really occur using powerfully diamagnetic substances such as pyrolytic graphite, in fact all materials can be weakly diamagnetic, teabags included. The degree to which this can be harnessed depends on the strength of the generated magnetic field, and so if we use an incredibly strong electromagnet, the teabag could be made to float on its opposing magnetic field. Get a strong enough magnet, and you can float next to it! This Wikipedia article has a pretty picture of a frog levitating diamagnetically: http://en.wikipedia.org/wiki/Diamagnetism#Diamagnetic_levitation