Who is Montresor talking to in the Cask of Amontillado?

He is talking to Fortunato his enemy. At other times, such as in the beginning, he is talking to some unnamed and unknown character in the story. In the beginning he says "You, who so well know the nature of my soul." It seems as if Montresor is talking to the reader as well, however at that point in the story, the reader knows nothing about Montresor.

Either way, this is a common device used by writers to create the setting of the story, introduce characters, set the mood and hook the reader into the plot and therefore into the story itself.