Why is your goldfish sluggish?

This will certainly depend on what you mean by irritated. If the fish is swimming quickly and flashing or rubbing against things in the tank, this can be a sign of parasites. It can also be a problem with the water quality, so this should be tested to check for the levels of ammonia, nitrites, and nitrates which should be 0, 0, and less than 10 respectively; if it is higher, you should do a 50% to 75% water change.