How much asbestos can kill a grown man?

Asbestos is not poisonous, or even chemically toxic, so it is impossible to say how much is dangerous.

In even microscopical trace amounts, as dust in the lungs, it has negative long-term effects.

Think about it like this:

Asbestos is like broken glass, that no matter how small you break it up still has pointy edges that cut you. It is light and fibrous, so it easily gets in your deeper lungs. And stays there, microscopically cutting and ittitating the lung tissues forever. You body's immune system tries and fails to handle these irritations, resulting in scar tissue formation. This causes shortness of breath, etc. The constant inflammation can also trigger other latent problems, such as cancer, etc.