Can you go to jail for taking someones credit card but not using it?

If you know the person and was going to give it back to them, then no. If you were planning on using it but got caught, then yes. Really it depends on the situation.