Skip to article frontmatterSkip to article content
Misc. Notes

Sensitivity Analysis - Gaussian Approximation

CSIC
UCM
IGEO

Context

We are given some approximations of inputs"

Uncertainty Inputs:x∼N(μx,Σx)Model:x=μx+εx,εx∼N(0,Σx)\begin{aligned} \text{Uncertainty Inputs}: && && \boldsymbol{x} &\sim \mathcal{N}(\boldsymbol{\mu_x},\boldsymbol{\Sigma_x}) \\ \text{Model}: && && \boldsymbol{x} &= \boldsymbol{\mu_x} + \boldsymbol{\varepsilon_x}, && && \boldsymbol{\varepsilon_x} \sim \mathcal{N}(0,\boldsymbol{\Sigma_x}) \end{aligned}
(1)

We have the data likelihood given

y∼p(y∣x,θ)\boldsymbol{y}\sim p(\boldsymbol{y}|\boldsymbol{x},\boldsymbol{\theta})
(2)

Formulation

We assume that this predictive density is given by a Gaussian distribution

y∼p(y∣x,θ)≈N(y∣μy,Σy) \begin{aligned} \boldsymbol{y} &\sim p(\boldsymbol{y}|\boldsymbol{x},\boldsymbol{\theta}) \approx \mathcal{N} \left(\boldsymbol{y}\mid \boldsymbol{\mu_y}, \boldsymbol{\Sigma_y}\right) \end{aligned} \\
(3)

where hμ\boldsymbol{h_\mu} and hσ\boldsymbol{h_\sigma} is the predictive mean and variance respectively. For example, this predictive mean could be a basis function, a non-linear function or a neural network. The predictive variance function could be constant or a simple linear function.

Predictive Mean:μy=hμ(x;θ),hμ:RDx×Θ→RDyPredictive Variance:Σy=hσ2(x;θ),hσ2:RDx×Θ→RDy×Dy\begin{aligned} \text{Predictive Mean}: && && \boldsymbol{\mu_y} &= \boldsymbol{h_\mu}(\boldsymbol{x};\boldsymbol{\theta}), && && \boldsymbol{h_\mu}: \mathbb{R}^{D_x}\times\mathbb{\Theta}\rightarrow\mathbb{R}^{D_y}\\ \text{Predictive Variance}: && && \boldsymbol{\Sigma_y} &= \boldsymbol{h_{\sigma^2}}(\boldsymbol{x};\boldsymbol{\theta}), && && \boldsymbol{h_{\sigma^2}}: \mathbb{R}^{D_x}\times\mathbb{\Theta}\rightarrow\mathbb{R}^{D_y\times D_y} \end{aligned}
(4)

We have ways to estimate these quantities as follows using the law of iterated expectations.

μy(x;θ)=Ex[hμ(x,θ)]σy(x;θ)=Ex[hσ2(x,θ)]+Ex[hμ2(x,θ)]−Ex2[hμ(x,θ)]\begin{aligned} \boldsymbol{\mu}_{\boldsymbol{y}}(\boldsymbol{x};\boldsymbol{\theta}) &= \mathbb{E}_{\boldsymbol{x}} \left[ \boldsymbol{h_\mu}(\boldsymbol{x},\boldsymbol{\theta})\right] \\ \boldsymbol{\sigma}_{\boldsymbol{y}}(\boldsymbol{x};\boldsymbol{\theta}) &= \mathbb{E}_{\boldsymbol{x}} \left[ \boldsymbol{h_{\sigma^2}}(\boldsymbol{x},\boldsymbol{\theta})\right] + \mathbb{E}_{\boldsymbol{x}} \left[ \boldsymbol{h_\mu}^2(\boldsymbol{x},\boldsymbol{\theta})\right] - \mathbb{E}_{\boldsymbol{x}}^2 \left[ \boldsymbol{h_\mu}(\boldsymbol{x},\boldsymbol{\theta})\right] \end{aligned}
(5)

In integral form, we can write this as:

μy(x;θ)=∫μx(x,θ)p(x)dxσy(x;θ)=∫hσ2(x,θ)p(x)dx+∫hμ2(x,θ)p(x)dx−[∫μx(x,θ)p(x)dx]2 \begin{aligned} \boldsymbol{\mu}_{\boldsymbol{y}}(\boldsymbol{x};\boldsymbol{\theta}) &= \int \boldsymbol{\mu_x}(\boldsymbol{x},\boldsymbol{\theta}) p(\boldsymbol{x})d\boldsymbol{x} \\ \boldsymbol{\sigma}_{\boldsymbol{y}}(\boldsymbol{x};\boldsymbol{\theta}) &= \int \boldsymbol{h_{\sigma^2}}(\boldsymbol{x},\boldsymbol{\theta}) p(\boldsymbol{x})d\boldsymbol{x} + \int \boldsymbol{h_\mu}^2(\boldsymbol{x},\boldsymbol{\theta})p(\boldsymbol{x})d\boldsymbol{x} - \left[\int \boldsymbol{\mu_x}(\boldsymbol{x},\boldsymbol{\theta}) p(\boldsymbol{x})d\boldsymbol{x}\right]^2 \end{aligned}
(6)

Taylor Approximation

Ex[hμ(x,θ)]≈hμ(μx,θ)+12Tr[∂2hμ(μx,θ)Σx]Ex[hμ(x,θ)]≈hμ(μx,θ)+12Tr[∂2hμ(μx,θ)Σx]Ex[hμ(x,θ)]≈hμ(μx,θ)+12Tr[∂2hμ(μx,θ)Σx]\begin{aligned} \mathbb{E}_{\boldsymbol{x}} \left[ \boldsymbol{h_\mu}(\boldsymbol{x},\boldsymbol{\theta})\right] &\approx \boldsymbol{h_\mu}(\boldsymbol{\mu_x},\boldsymbol{\theta}) + \frac{1}{2}\text{Tr}\left[ \partial^2\boldsymbol{h_\mu}(\boldsymbol{\mu_x},\boldsymbol{\theta}) \boldsymbol{\Sigma_x} \right] \\ \mathbb{E}_{\boldsymbol{x}} \left[ \boldsymbol{h_\mu}(\boldsymbol{x},\boldsymbol{\theta})\right] &\approx \boldsymbol{h_\mu}(\boldsymbol{\mu_x},\boldsymbol{\theta}) + \frac{1}{2}\text{Tr}\left[ \partial^2\boldsymbol{h_\mu}(\boldsymbol{\mu_x},\boldsymbol{\theta}) \boldsymbol{\Sigma_x} \right] \\ \mathbb{E}_{\boldsymbol{x}} \left[ \boldsymbol{h_\mu}(\boldsymbol{x},\boldsymbol{\theta})\right] &\approx \boldsymbol{h_\mu}(\boldsymbol{\mu_x},\boldsymbol{\theta}) + \frac{1}{2}\text{Tr}\left[ \partial^2\boldsymbol{h_\mu}(\boldsymbol{\mu_x},\boldsymbol{\theta}) \boldsymbol{\Sigma_x} \right] \end{aligned}
(7)

Moment-Matching

Sensitivity Analysis
Problem Formulation
Algorithms
PCA