How do you unmatt a dogs hair?

If the mats are small, you can very gently pull them out. You have to be patient or else you'll hurt the rabbit. If you tear the rabbit's skin, this can lead to infection, cyst, abscess, etc. A single small mat can take a few days' work. This works much better when the rabbit is moulting and the under-fur is already loose.

If there are many mats or big mats, you should go to a vet for help. The vet will cut them out and then shave the fur. The rabbit might need to be anaesthetised if the mats are really bad; if not, then one person can restrain the rabbit while another person takes out the mats.

Do not cut or shave your rabbit's fur at home. Rabbit skin is very thin and very stretchy: even vets have been known to accidentally cut it. When you do cut it, it can quickly stretch to an alarming size!

On the other hand, you can't just leave the mats, because they'll pull at the skin, causing irritation and even possibly injury.

It's not normal for a rabbit to have mats in their fur. Only if they're sick, or have mobility problems (like Arthritis), or are living in unhealthy conditions will they develop mats. Rabbits are clean animals and groom themselves. In addition to the problem (the mats), you should treat the cause!