I had my tongue pierced 5 days ago. I noticed white discharge coming from the bottom. There is no swelling only a little stinging feeling. Is this normal?

Yes. That is probably pus. It's probably a bit infected in its healing, but it will happen every so often. It should go away with proper care. Make sure you are still cleansing well.

If it doesn't sort itself out in the next day or so, you should see a doctor.