Do you have to pay damage on a loaner from the dealer?

Yup ... your insurance should cover the damages. The dealer lent you a car in good faith stipulating that the vehicle is returned on the date and time agreed and in the same condition.