Are aquatic aquarium frogs considered invertebrates?

No they are still vertebrates. And just to intrigue you further, though tadpoles might not appear to have a vertebrae, they too have one.