Bayesian Quadrature: Mean Derivation
====================================

We have:

.. math:: \mathbb{E}[Z]= \int\ell(x)p(x)\ \mathrm{d}x.

We approximate this as:

.. math:: \mathbb{E}[Z]\approx \int \bar{\ell}(x)p(x)\ \mathrm{d}x,

where the function :math:`\bar{\ell}` is the mean of the Gaussian
process over :math:`\exp(\log\ell)`.

We expand the first term to obtain:

.. math::


   \begin{align*}
   E[m_\ell|x_s] &= \int K_{\exp(\log\ell)}(x,\mathbf{x}_c)K_{\exp(\log\ell)}(\mathbf{x}_c,\mathbf{x}_c)^{-1}\bar{\ell}(\mathbf{x}_c)p(x)\ \mathrm{d}x\\\\
   &= \left(\int K_{\exp(\log\ell)}(x,\mathbf{x}_c)p(x)\ \mathrm{d}x\right)\ K_{\exp(\log\ell)}(\mathbf{x}_c,\mathbf{x}_c)^{-1}\bar{\ell}(\mathbf{x}_c)\\\\
   &= \left(\int K_{\exp(\log\ell)}(x, \mathbf{x}_c)p(x)\ \mathrm{d}x\right)\ \alpha(\mathbf{x}_c)
   \end{align*}

where
:math:`\alpha(\mathbf{x}_c) = K_{\exp(\log\ell)}(\mathbf{x}_c, \mathbf{x}_c)^{-1}\bar{\ell}(\mathbf{x}_c)`.

Assuming :math:`K_{\exp(\log\ell)}` is a Gaussian kernel, and
:math:`p(x)` is a Gaussian density with mean :math:`\mu` and covariance
:math:`\Sigma`, the integral can be expressed analytically as follows,
from [O13]\_:

.. math::


   \begin{align*}
   \int K_{\exp(\log\ell)}(x, \mathbf{x}_c)p(x)\ \mathrm{d}x&=\int h_\ell^2 \mathcal{N}\left(\mathbf{x}_c\ \big\vert\  x, W_\ell\right)\mathcal{N}\left(x\ \big\vert\  \mu, \Sigma\right)\ \mathrm{d}x\\\\
   &= h_\ell^2 \mathcal{N}\left(\mathbf{x}_c\ \big\vert\  \mu, W_\ell + \Sigma\right)
   \end{align*}