Bayesian Quadrature: Expected Variance
The equation for the expected variance is given by Equation 13 of
[OD12]_:
\[\begin{align*}
E[V_Z|\log\ell_{s,a}] = S(Z\ |\ \log\ell_{s}) - \int m(Z\ |\ \log\ell_{s,a})^2 \mathcal{N}\left( \log\ell_a \ \big\vert\ \hat{m}_a, \hat{C}_a\right)\ \mathrm{d}\log\ell_a
\end{align*}\]
From before, we have:
\[\begin{split}\begin{align*}
m(Z\ |\ \log\ell_{s,a})&=E[m_\ell|x_{s,a}]\\\\
&= h_\ell^2 \mathcal{N}\left(\mathbf{x}_{c,a}\ \big\vert\ \mu, W_\ell + \Sigma\right)K_{\exp(\log\ell)}(\mathbf{x}_{c,a}, \mathbf{x}_{c,a})^{-1}\bar{\ell}(\mathbf{x}_{c,a})
\end{align*}\end{split}\]
For brevity, let:
\[A_{c,a} = h_\ell^2 \mathcal{N}\left(\mathbf{x}_{c,a}\ \big\vert\ \mu, W_\ell + \Sigma\right)K_{\exp(\log\ell)}(\mathbf{x}_{c,a}, \mathbf{x}_{c,a})^{-1}\]
such that
\(m(Z\ |\ \log\ell_{s,a})=A_{c,a}\bar{\ell}(\mathbf{x}_{c,a})\).
We have that
\(\int\exp(cy)\mathcal{N}\left( y \ \big\vert\ m, S \right)=\exp(cm + \frac{1}{2}c^2 S)\),
which allows us to marginalize out \(\bar{\ell}_a\). Thus, we need
to break apart
\(\bar{\ell}(\mathbf{x}_{c,a})=[\bar{\ell}_c; \bar{\ell}_a]\),
giving:
\[\begin{split}\begin{align*}
m(Z\ |\ \log\ell_{s,a})^2 &= \left([A_c, A_a][\bar{\ell}_c; \bar{\ell}_a]\right)^2\\\\
&= \left( A_c\bar{\ell}_c + A_a\bar{\ell}_a \right)^2\\\\
&= (A_c\bar{\ell}_c)^2 + 2(A_c\bar{\ell}_c)(A_a\bar{\ell}_a) + (A_a\bar{\ell}_a)^2
\end{align*}\end{split}\]
Plugging this into the expected squared mean:
\[\begin{split}\begin{align*}
E[m_Z^2|\log\ell_{s,a}] &= \int m(Z\ |\ \log\ell_{s,a})^2 \mathcal{N}\left( \log\ell_a \ \big\vert\ \hat{m}_a, \hat{C}_a\right)\ \mathrm{d}\log\ell_a\\\\
&= (A_c\bar{\ell}_c)^2 + \int \left( 2(A_c\bar{\ell}_c)(A_a\bar{\ell}_a) + (A_a\bar{\ell}_a)^2 \right)p(\log\ell_a) \ \mathrm{d}\log\ell_a\\
&= (A_c\bar{\ell}_c)^2 + 2A_c\bar{\ell}_cA_a\int \bar{\ell}_ap(\log\ell_a)\ \mathrm{d}\log\ell_a + A_a^2\int \bar{\ell}_a^2 p(\log\ell_a) \ \mathrm{d}\log\ell_a\\
&= (A_c\bar{\ell}_c)^2 + 2A_c\bar{\ell}_cA_a\int \exp(\log\ell_a)p(\log\ell_a)\ \mathrm{d}\log\ell_a + A_a^2\int \exp(2
\log\ell_a) p(\log\ell_a) \ \mathrm{d}\log\ell_a
\end{align*}\end{split}\]
Using the identity
\(\int\exp(cy)\mathcal{N}\left( y \ \big\vert\ m, S \right)=\exp(cm + \frac{1}{2}c^2 S)\),
we have:
\[E[m_Z^2|\log\ell_{s,a}] = A_c^2\bar{\ell}_c^2 + 2A_c\bar{\ell}_cA_a\exp\left(\hat{m}_a + \frac{1}{2}\hat{C}_a\right) + A_a^2\exp\left(2\hat{m}_a + 2\hat{C}_a\right)\]
Putting it all together, we obtain:
\[\begin{split}\begin{align*}
E[V_Z|\log\ell_{s,a}] &= S(Z\ |\ \log\ell_{s}) - \left( A_c^2\bar{\ell}_c^2 + 2A_c\bar{\ell}_cA_a\exp\left(\hat{m}_a + \frac{1}{2}\hat{C}_a\right) + A_a^2\exp\left(2\hat{m}_a + 2\hat{C}_a\right)\right)\\\\
&= V_Z + m_Z^2 - \left( A_c^2\bar{\ell}_c^2 + 2A_c\bar{\ell}_cA_a\exp\left(\hat{m}_a + \frac{1}{2}\hat{C}_a\right) + A_a^2\exp\left(2\hat{m}_a + 2\hat{C}_a\right)\right)
\end{align*}\end{split}\]
where \(S(Z\ |\ \log\ell_s) = V_Z + m_Z^2\) from Equation 9 of
[OD12]_.