Decision Theory

Table of Contents

Risk

Posterior risk:

\begin{equation*} \begin{split} r \big( \hat{\theta} \mid x \big) &= \int L \big( \theta, \hat{\theta}(x) \big) f( \theta \mid x ) \ d \theta \\ &= \mathbb{E}_{\theta \mid X} \Big[ L \big( \theta, \hat{\theta}(X) \big) \Big] \end{split} \end{equation*}

(Frequentist risk):

\begin{equation*} R(\theta, \hat{\theta}) = \int L \big( \theta, \hat{\theta}(x) \big) f(x \mid \theta) \ dx = \mathbb{E}_{x \mid \theta} \Big[ L \big( \theta, \hat{\theta}(X) \big) \Big] \end{equation*}

Bayesian risk:

\begin{equation*} r \big( f, \hat{\theta} \big) = \iint L \big( \theta, \hat{\theta}(x) \big) f(x, \theta) \ dx d \theta = \mathbb{E}_{\theta, X} \Big[ L \big( \theta, \hat{\theta}(X) \big) \Big] \end{equation*}

Covariate shift

We assume that

\begin{equation*} P_s(Y \mid X = x) = P_t( Y \mid X = x), \quad \forall x \in \mathcal{X} \end{equation*}

but

\begin{equation*} P_s(X) \ne P_t(X) \end{equation*}

The difference between the two domains is called covariate shift.