Why is Hrothgar safe from Grendel's attack?

Grendel only ever attacks the hall, never any other building. Hrothgar and Wealhtheow retire to a bower (buhr) for the night. Beowulf similarly is not in the hall when Grendel's mother attacks.