Bayesian Quadrature: Mean Derivation
We have:
\[\mathbb{E}[Z]= \int\ell(x)p(x)\ \mathrm{d}x.\]
We approximate this as:
\[\mathbb{E}[Z]\approx \int \bar{\ell}(x)p(x)\ \mathrm{d}x,\]
where the function \(\bar{\ell}\) is the mean of the Gaussian
process over \(\exp(\log\ell)\).
We expand the first term to obtain:
\[\begin{split}\begin{align*}
E[m_\ell|x_s] &= \int K_{\exp(\log\ell)}(x,\mathbf{x}_c)K_{\exp(\log\ell)}(\mathbf{x}_c,\mathbf{x}_c)^{-1}\bar{\ell}(\mathbf{x}_c)p(x)\ \mathrm{d}x\\\\
&= \left(\int K_{\exp(\log\ell)}(x,\mathbf{x}_c)p(x)\ \mathrm{d}x\right)\ K_{\exp(\log\ell)}(\mathbf{x}_c,\mathbf{x}_c)^{-1}\bar{\ell}(\mathbf{x}_c)\\\\
&= \left(\int K_{\exp(\log\ell)}(x, \mathbf{x}_c)p(x)\ \mathrm{d}x\right)\ \alpha(\mathbf{x}_c)
\end{align*}\end{split}\]
where
\(\alpha(\mathbf{x}_c) = K_{\exp(\log\ell)}(\mathbf{x}_c, \mathbf{x}_c)^{-1}\bar{\ell}(\mathbf{x}_c)\).
Assuming \(K_{\exp(\log\ell)}\) is a Gaussian kernel, and
\(p(x)\) is a Gaussian density with mean \(\mu\) and covariance
\(\Sigma\), the integral can be expressed analytically as follows,
from [O13]_:
\[\begin{split}\begin{align*}
\int K_{\exp(\log\ell)}(x, \mathbf{x}_c)p(x)\ \mathrm{d}x&=\int h_\ell^2 \mathcal{N}\left(\mathbf{x}_c\ \big\vert\ x, W_\ell\right)\mathcal{N}\left(x\ \big\vert\ \mu, \Sigma\right)\ \mathrm{d}x\\\\
&= h_\ell^2 \mathcal{N}\left(\mathbf{x}_c\ \big\vert\ \mu, W_\ell + \Sigma\right)
\end{align*}\end{split}\]