Is this sentence correct Although Mark and my visit was brief I am so grateful to have spent the time with you?

No. The structurally correct form is "Mark's and my visit" because the possessive must extend to Mark as it would if there were no compound subject. Ordinarily you would avoid the problem and say "our" instead of "Mark's and my."