Do dying fire alarms make a loud beep sound?

yes

They usually beep repeatedly, once every few seconds or once a minute to indicate the batteries are going dead. You should replace the batteries immediately.