Can cactus fruits be poisonous

No, cactus fruits aren't poisonous. The concern is flavorability, not poisonousness. Some of the fruits are more flavorful than others. Some fruits are more flavorful when they're a certain color. For example, in the range of colors from green to purple and red of the prickly pear cactus [Opuntia spp], the darker colored fruits are considered less tasty.