Are math enthusiasts generally poor in English?

No. They're generally good all-around students.

Answer

Interestingly, though, the opposite can be true. Verbal skills are closely linked to being able to remember steps of a procedure, correcting errors and remembering where you are in the process.