What side of the table do you clear from?

The "side of the diner" - not "side of the table".

One serves food by standing to the right of the person one is serving, and one clears by removing plates and cutlery when standing on the left of the person dining.