How did the curse fall upon lady of shalott?

the poem doesn't exactly say who cursed the lady of shalott but it does say:

'She has heard a whisper say,

A curse is on her if she stay

To look down to Camelot.'

so basically i think that some whispers said it to her either in her head or whatever but it was whispers