Is it he was seen by yourself or seen by you grammar?

Yes, you could say he was seen by yourself, or he was seen by you, but the simpler way to express that idea would be, you saw him. In most cases, a simpler phrasing will work better.