Is the burning sore feeling you get in your arms from doing something such as window washing considered muscle fatigue?

That's what it is. It is a combination of lactic acid and your muscles have broken down from the overuse. Rest them for a day and the muscle will heal itself and be stronger.