Why do the sailors think the albatross follows the ship?

Because the mariner had shot the bird, they thought he had brought bad luck upon them. As punishment he had to wear the dead bird around his neck, a symbol of his guilt.