Skip to content

Error Propagation

Taylor Series Expansion

A Taylor series is representation of a function as an infinite sum of terms that are calculated from the values of the functions derivatives at a single point - Wiki

Often times we come across functions that are very difficult to compute analytically. Below we have the simple first-order Taylor series approximation.

Let's take some function f(\mathbf x) where \mathbf{x} \sim \mathcal{N}(\mu_\mathbf{x}, \Sigma_\mathbf{x}) described by a mean \mu_\mathbf{x} and covariance \Sigma_\mathbf{x}. The Taylor series expansion around the function f(\mathbf x) is:

\mathbf z = f(\mathbf x) \approx f(\mu_{\mathbf x}) + \frac{\partial f}{\partial \mathbf x} \bigg\vert_{\mathbf{x} = \mu_\mathbf{x}}\left( \mathbf x - \mu_{\mathbf x} \right)

Law of Error Propagation

This results in a mean and error covariance of the new distribution \mathbf z defined by:

\mu_{\mathbf z} = f(\mu_{\mathbf x})
\Sigma_\mathbf{z} = \nabla_\mathbf{x} f(\mu_{\mathbf x}) \; \Sigma_\mathbf{x} \; \nabla_\mathbf{x} f(\mu_{\mathbf x})^{\top}

Proof: Mean Function

Given the mean function:

\mathbb{E}[\mathbf{x}] = \frac{1}{N} \sum_{i=1} x_i

We can simply apply this to the first-order Taylor series function.

\begin{aligned} \mu_\mathbf{z} &= \mathbb{E}_{\mathbf{x}} \left[ f(\mu_{\mathbf x}) + \frac{\partial f}{\partial \mathbf x} \bigg\vert_{\mathbf{x} = \mu_\mathbf{x}}\left( \mathbf x - \mu_{\mathbf x} \right) \right] \\ &= \mathbb{E}_{\mathbf{x}} \left[ f(\mu_{\mathbf x}) \right] + \mathbb{E}_{\mathbf{x}} \left[ \frac{\partial f}{\partial \mathbf x} \bigg\vert_{\mathbf{x} = \mu_\mathbf{x}}\left( \mathbf x - \mu_{\mathbf x} \right) \right] \\ &= f(\mu_{\mathbf x}) + \mathbb{E}_{\mathbf{x}} \left[ \frac{\partial f}{\partial \mathbf x} \bigg\vert_{\mathbf{x} = \mu_\mathbf{x}} \mathbf x \right]- \mathbb{E}_{\mathbf{x}} \left[ \frac{\partial f}{\partial \mathbf x} \bigg\vert_{\mathbf{x} = \mu_\mathbf{x}}\mu_{\mathbf x} \right] \\ &= f(\mu_{\mathbf x}) + \frac{\partial f}{\partial \mathbf x} \bigg\vert_{\mathbf{x} = \mu_\mathbf{x}} \mu_\mathbf{x} - \frac{\partial f}{\partial \mathbf x} \bigg\vert_{\mathbf{x} = \mu_\mathbf{x}}\mu_{\mathbf x} \\ &= f(\mu_{\mathbf x}) \\ \end{aligned}

Proof: Variance Function

Given the variance function

\mathbb{V}[\mathbf{x}] = \mathbb{E}\left[ \mathbf{x} - \mu_\mathbf{x} \right]^2
\begin{aligned} \sigma_\mathbf{z}^2 &= \mathbb{E} \left[ f(\mu_\mathbf{x}) - \frac{\partial f}{\partial \mathbf{x}} \bigg\vert_{\mathbf{x}=\mu_\mathbf{x}} (\mathbf{x} - \mu_\mathbf{x}) - \mu_\mathbf{x} \right] \\ &= \mathbb{E} \left[ \frac{\partial f}{\partial \mathbf{x}} \bigg\vert_{\mathbf{x}=\mu_\mathbf{x}} (\mathbf{x} - \mu_\mathbf{x})\right]^2 \\ &= \left( \frac{\partial f}{\partial \mathbf{x}} \bigg\vert_{\mathbf{x}=\mu_\mathbf{x}} \right)^2 \mathbb{E}\left[ \mathbf{x} - \mu_\mathbf{x}\right]^2\\ &= \left( \frac{\partial f}{\partial \mathbf{x}} \bigg\vert_{\mathbf{x}=\mu_\mathbf{x}} \right)^2 \Sigma_\mathbf{x} \end{aligned}

I've linked a nice tutorial for propagating variances below if you would like to go through the derivations yourself.


Resources

  • Essence of Calculus, Chapter 11 | Taylor Series - 3Blue1Brown - youtube
  • Introduction to Error Propagation: Derivation, Meaning and Examples - PDF
  • Statistical uncertainty and error propagation - Vermeer - PDF