How do you get rid of stubborn band-aid residue?

I have tried nail polish remover and it did not work. You can let it go away on it's own but you can also get most of it off by gently rubbing a wet towel on and around it.