Why is numerical differentiation unstable as an approximation method?

it is unstable as an approximation method due to round-off errors involved and the fact that it involves taking a limit of h->0. Doing this for an iteration method is a really bad idea because as h gets very close to 0, say h=1x10-10, f(x0+h) and f(x0) are virtually the same number and subtracting near equal numbers in an iteration formula has the effect of producing huge errors and as such would not give a very reliable answer.

In fact if they are to close the computer can't recognise a difference between the two when doing the algorithm and you end up with 0/h so you don't get an answer at all! It is because of these reasons that taking the formula for numerical differentiation as it is, with the limit of h-> 0, is unstable and not suitable as an approximation method.

It is however, a good place to start. What we do instead is take the formula, but get rid of the limit and make it so we take h at very small values but not so small as to create huge round errors, so values such as h=0.1 or 0.001 or something like that. Then to get the answer we obtain from the formula more accurate, we apply a truncation error to it. i.e. we subtract a small amount from it to get it closer to the answer. This error correction term is f''([xi])|h|/2 where x0<[xi]

f'(x0)=(f(x0+h)-f(x0))/h - (h/2)*f''([xi])

This is known as the forward-difference formula if h>0and the backward difference formula if h<0