Why is Australia the oldest continent?

Because of the lack of tectonic activity, compared to the rest of the continents. Which means features from early parts of Earths history have managed to survive, and have not been destroyed in late-Tertiary and Quaternary uplift and extensive glacial erosion that the other continents went through.