How do you replace the passenger side power mirror assembly on a 1997 Toyota Camry?

You have to remove the door panel. And the panel covering the nuts (3 of them) attaching the mirror to the door (it's a triangle piece of plastic). If you have the protection behind the door panel you will have to remove it (don't damage it, it's attached with a special glue which can be reused) in order to disconnect the mirror wires. Remove the nuts, remove the mirror. You might have to force it because glue is used in between the door and mirror. Installation is reversed to disassembly.

PS: when you are installing the replacement make sure that you keep the mirror safely all the time. It's a common mistake, people do not hold it and of course brake it.