How easy were castles to defend?

To answer your question, the Norman castles had deep ditches and high embankments, which shows how hard it was to attack. It was also quite easy to defend as look-out posts at the top of the bailey could give the inhabitants an early warning if there were signs of a rebellion among the locals.