Can you eat mushrooms if your allergic to penicillin?

I'm allergic to penicillin only, and have done mushrooms twice before. But 1st time with Pizza, second time shredded into candy, but if I eat it straight could it affect me any differently?