Is calling someone white racist

Not necessarily, though some may take offense. Skin color is just a part of someone's physical description, so it should not automatically be taken as racist. Really, it depends on why and how you call someone 'white'.

The reason nonwhite races are offended by terms related to their skin color is because white people have been the dominant race in the past and subjected others to bad treatment (e.g. slavery). No race has enslaved white people. Calling someone what is only racist if you say it with malice implying they are below you.