Do you capitalize first baseman

Not precisely.

If you're preferring to the type of sport, or the type of ball itself, then no.

If a club name has the Word in it, then yes. For example: Major League Baseball.