Importance Sampling

Importance Sampling is a technique for estimating the expectation $\mu$ of a random variable $f(x)$ under distribution $p$ from samples of a different distribution $q$.

The key observation is that $\mu$ is can expressed as the expectation of a different random variable \(f^{\star}(x)=\frac{p(x)}{q(x)} f(x)\) under $q$

Technique condition: $q$ must have support everywhere $p$ dose. Without this condition, the equation is biased! Note: $q$ can support things that $p$ doesn't.