Is going to whence

No. Whence means from what place, the opposite of going to. Whence can also mean from what cause or what origin

Whence have you come?

And where was this foreign land, whence the conquerors would come?