13) Stability

13) Stability#

Last time#

  • Newton’s method

  • Convergence of fixed-point method

  • Different formulation of Newton’s method

Today#

  1. Forward and backward stability

  2. Recap and Q/A

using Plots
default(linewidth=4, legendfontsize=12)

1. Stability#

(Forward) Stability#

“nearly the right answer to nearly the right question”

\[ \frac{\lvert \tilde f(x) - f(\tilde x) \rvert}{| f(\tilde x) |} \in O(\epsilon_{\text{machine}}) \]
for some \(\tilde x\) that is close to \(x\)

Backward Stability#

“exactly the right answer to nearly the right question”

\[ \tilde f(x) = f(\tilde x) \]
for some \(\tilde x\) that is close to \(x\)

Note:

  • Every backward stable algorithm is (\(\implies\)) stable.

  • Not every stable algorithm is backward stable.

  • The difference is in the focus: forward analysis is concerned with what the method reaches, while backward analysis looks at the problem being solved (which is why we can speak of ill-conditioned methods and ill-conditioned problems).

  • In a backward stable algorithm the errors introduced during the algorithm have the same effect as a small perturbation in the data.

  • If the backward error is the same size as any uncertainty in the data then the algorithm produces as good a result as we can expect.

Reading: What is Numerical Stability?#

Accuracy of backward stable algorithms (Theorem)#

A backward stable algorithm for computing \(f(x)\) has relative accuracy

\[ \left\lvert \frac{\tilde f(x) - f(x)}{f(x)} \right\rvert \lesssim \kappa(f) \epsilon_{\text{machine}} . \]
Backward stability is generally the best we can hope for.

In practice, it is rarely possible for a function to be backward stable when the output space is higher dimensional than the input space.

2. Recap and Q/A#

Contents