Why is it so important that roger chillingworth knows who's the babies father is in the scarlet letter.?

So he could punish the man who stole his wife. Once he finds out who it is he puts them through the suffering he had to go through and makes it even worse for him.