Why are the memories of the giver important?

The memories of the Giver are important, theoretically, so that they still have one touchstone to their past, and can have warning if some lesson of the past needs to teach them something, even though they have moved beyond that.

Truthfully though, the memories without appropriate context probably couldn't have helped that society much, because any lesson the giver tried to share wouldn't be understood by people who had been raised to not even understand the basics of what he was saying.

Later, after the giver escapes (in later books), his memories are valuable to people who are trying to fight scary societies like the one he grew up in.