Inverse Function Theorem

Resources: * Wiki * YouTube * Prof Ghist Math - Inverse Function Theorem * The Infinite Looper - Inv Fun Theorem * Professor Leonard - Fundamental Theorem of Calculus | Derivatives of Inverse Functions

Source:

• Mathematics for Machine Learning - Deisenroth (2019)
• Change of Variables: A Precursor to Normalizing Flow - Rui Shu
• Pattern Recognition and Machine Learning - Bishop (2006)

Often we are faced with the situation where we do not know the distribution of our data. But perhaps we know the distribution of a transformation of our data, e.g. if we know that $X$ is a r.v. that is uniformly distributed, then what is the distribution of $X^2 + X + c$? In this case, we want to understand what's the relationship between the distribution we know and the transformed distribution. One way to do so is to use the inverse transform theorem which directly uses the cumulative distribution function (CDF).

Let's say we have $u \sim \mathcal U(0,1)$ and some invertible function $f(\cdot)$ that maps $X \sim \mathcal P$ to $u$.

$$x = f(u)$$

Now, we want to know the probability of $x$ when all we know is the probability of $u$.

$$\mathcal P(x)=\mathcal P(f(u)=x)$$

So solving for $u$ in that equation gives us:

$$\mathcal P(x) = \mathcal P(u=f^{-1}(x))$$

Now we see that $u=f^{-1}(x)$ which gives us a direct formulation for moving from the uniform distribution space $\mathcal U$ to a different probability distribution space $\mathcal P$.

Probability Integral Transform

Resources * Brilliant

Does a nice example where they talk about the problems with fat-tailed distributions. * Wiki * CrossValidated * How does the inverse transform method work * Help me understand the quantile (inverse CDF) function * Youtube * Ben Hambert - Intro to Inv Transform Sampling * Mathematical Monk * Intro | General Case | Invertible Case * Code Review - Inverse Transform Sampling * R Markdown - Inverse Transform Sampling * Using Chebyshev - Blog | Code * CDFs - Super powerful way to visualize data and also is uniformly distriuted * Histograms and CDFs - blog * Why We Love CDFS so Much and not histograms - Blog * Boundary Issues * Confidence Band from DKW inequality - code * Make Monotonic - code * Matplotlib example of CDF bins versus theoretical (not smooth) - Code * Alternatives * KDE * Statsmodels Implementation - code | Univariate * KDE vs Histograms - blog * Empirical CDF * The Empirical Distribution Function - blog * Plotting an Empirical CDF In python - blog * Scipy histogram - code * Empirical CDF Function - code * ECDFs - notebook

Derivative of an Inverse Function

• MathInsight - Link