Can organic fertilizer leach out of the soil and enter the water?

Certainly. overwatering or over fertilization can cause the nutrients to wash away and into a watershed. Results: higher nitrogen levels in the water so more fuel for the algae to feed on. Keep applications of fertilizer down to 1/2lb of nitrogen per 1000 sq ft applied and water sensibly, repeat in two weeks if your desired effects are not noticeable, that way the nutrients can be used by the plant and are not excess all at once such as in a 11/2 or 2lb per 1000 app would be. Dont worry too much about the release of harmful chemicals though since it is an organic, the unintentional raising of the normal levels of these nutrients will be the only result which in all likelyhood will only raise the activity of aquatic plant growth.