Do baby red eared slider turtles have to have a heat lamp?

Yes, all captive turtles, especially babies, need lamps that provide both heat, and UVA and UVB rays. It is very important that you have both. If you don't have these lamps, your turtles may develop shell rot, which causes discoloration and the shell to become "squishy".

The heat lamp can be just a standard incandescent bulb, as long as it heats up the basking area enough. The UVA/UVB lamp isn't as important as heat, but both are necessary for the turtle.