Sensitivity Analysis - Problem Formulation

Context¶

Modeling¶

Image, we have some data

We are interested in finding the joint distribution which maps the co-variates, x, to the observations, y. We can decompose the joint distribution as follows

This can be done by finding the posterior parameters

There are a myriad of methods to find the parameters. For example we could use some conjugate methods if the functions are linear or we can use approximate inference or sampling. Irrespective of the method, we will have some set of parameters that we think are good for the model.

Post-Modeling Analysis¶

However, assuming we have found the parameters, we are interested in performing sensitivity analysis which is whereby we take a distribution of interest of our input co-variates, x, and we perform some analysis on the generated outputs.

So, our new problem is to find some sort of expectation over our data likelihood given our model parameters

In general, there are two ways we can tackle this: 1) deterministic inference and 2) stochastic inference. Deterministic inference is well known within the community as approximations. Stochastic inference is more known as MonteCarlo inference or some variant of.

Exact Inference¶

This is the case when we have linear methods and simple, conjugate distributions. If these two conditions are satisfied, then we can do a series of linear algebra tricks and identities to construct a closed-form solution to this integral problem.

Example¶

In this example, we can predict the mean and the covariance under a linear Gaussian model.

Parameters

# input covariates
m: Array["Dx"] = ...       # prior mean covariate, x
S: Array["Dx Dx"] = ...    # prior covariance covariate, x
# parameters
W: Array["Dy Dx"] = ...    # weight matrix
b: Array["Dy"] = ...       # bias vector
Q: Array["Dy Dy"] = ...    # observation covariance matrix

Prediction Function

y_mu: Array["Dy"] = W @ mu_x + b
y_cov: Array["Dy Dy"] = W @ sigma_x @ W.T + Q

Sample Function

# create covariate distribution
mvn_dist_x: Dist = MVM(μ_x, σ_x)
# sample covariates
n_samples: int = 100
seed: RNGKey = RNGKey(123)
x_samples: Array["Nx Dx"] = sample(dist=mvn_dist_x, seed=seed, shape=(n_samples,))
# calculate variables
μ_y: Array["Nx Dy"] = einsum("ND,D->ND", x, w) + b
σ_y: Array["Dy Dy"] = ...
# create observation distribution
mvn_dist_y: Dist = BatchedMVM(μ_y, σ_y)
# sample observations
n_samples: int = 100
seed: RNGKey = RNGKey(123)
y_samples: Array["Ny Nx Dy"] = sample(dist=mvn_dist_y, seed=seed, shape=(n_samples,))

Determinstic Inference¶

In general, we can do deterministic inference using

Some example methods include:

• Linearization, i.e., Taylor Expansion
• Unscented Transformation, i.e., Sigma Points
• Moment-Matching, i.e., GH-Quadrature, Bayesian Quadrature, etc.

Example¶

Stochastic Inference¶

Some example methods include:

• Sequential Monte Carlo
• Ensemble Points

We take a Gaussian potential on a new observation with an arbitrary likelihood given functions for conditional moments and make a Gaussian approximation. The equation is given as so: