Should the phrase out of plumb be hyphenated?

Yes and no. The out-of-plumb wall should be hyphenated while the wall was out of plumb should not.