Nice speaking with you or nice speaking to you?

Honestly, both are polite and correct. I would use "nice speaking with you" over the phone and "nice speaking to you" in person. Because over the phone your speaking with something. In person your talking directly to someone.