Can you be infected just by touching posion ivy?

Yes, that is the main way to catch Poison Ivy. You can even catch it just by being near the plants if you're highly allergic.