Does removing the plug from an 'off' socket save power?

No since it's off no power is going into it anyway. So it won't make a difference