Do you need your social security card with you to apply for a loan?

Yes. The business providing the car loan must run a credit check and that requires a social security number. While giving a retailer your social security number is optional, it isn't when trying to obtain credit.