How do you get rid of your ex girlfriend's current boyfriend?

If the ex girlfriend won't back off try getting your boyfriend to talk to her (maybe with you there also if you feel insecure about it) telling her that he is with you now and that she needs to get over him and move on.

Maybe she just needs that final ultimate confirmation that there is nothing left between them and they are not going to get back together before she can move on. Be gentle with her, as her feelings are obviously still raw.