Can you just give them your social security number to get a learners permit or do you have to have your social security card?

You will be asked to produce VALID identification. Contact/call your local DMV office to see what they require. Most (all?) states now offer the option of issuing you what is called a 'soundex' number instead of uisng your SSN on the permit.