Are all odd square roots the square roots of odd numbers?

Yes, they are.

Sketch of proof:

Odd square root is a number x of form x = 2n + 1.

Square of this root is:

x2=(2n+1)2=4n2+4n+1=y,

which can be expressed as:

y=2(2n2+2n)+1.

Let t = 2n2+2n.

Then,

y=2t+1, which is an odd number.