Does Ashfur get mad and try to kill Brambleclaw when Squirrelflight said she doesn't love him in the books Warriors?

No, but he tries to kill Hollyleaf, Jayfeather, and Lionblaze because he thought that if Squirrelflight watched her kits die then she would go to Ashfur.

But Hollyleaf, Jayfeather, and Lionblaze are not Squirrelflight and Brambleclaw's kits.