Why is zero not a factor?

Zero is only a factor of zero.

If I have 0 and multiply it by any other number, I still get 0.

No number can be divided by zero - the concept is meaningless.