Is it correct saying nice spoken to you over the phone?

The correct phrase would be "It was nice speaking to you over the phone." The verb "speaking" is used in the gerund form to indicate the action of conversation. The phrase conveys a polite sentiment after a phone call.