Do you say it was a pleasure to meet you and X or X and yourself?

In this particular situation, it really doesn't matter which order you use, the meaning is the same. When you are talking about yourself, it is another matter. Conventionally, you mention yourself last, as a sign of humility. It was kind of you to invite Fred and me. But if you are talking about two other people, then the sequence doesn't mean anything.