Is it true that fish only grow to their environment?

No it is not true. Fish, just like any other living thing, have a genetic imperative to grow to their full size. The reason people think they grow to their environment is because once the animal runs out of room to grow, it gets so stressed that it dies. This can take a long time. Goldfish, for example, need to have 20 gallons each to grow to their full size without stress, and can live for up to 25 years under ideal conditions! Most live 2-3 years, though, due to poor environment/lack of space.