What would a scientist have to prove to show that the organism they had discovered was a new species?

They would have to prove that it is new and unlike any other known organism. Hundreds of measurements, thousands of comparisons, and the opinions of other scientists would be needed to prove anything.