Is a staff of labourers correct?

No, the correct term is "staff of laborers." The phrase refers to a group of workers who are employed for manual labor tasks.