Bayesian Quadrature: Expected VarianceΒΆ

The equation for the expected variance is given by Equation 13 of [OD12]_:

\[\begin{align*} E[V_Z|\log\ell_{s,a}] = S(Z\ |\ \log\ell_{s}) - \int m(Z\ |\ \log\ell_{s,a})^2 \mathcal{N}\left( \log\ell_a \ \big\vert\ \hat{m}_a, \hat{C}_a\right)\ \mathrm{d}\log\ell_a \end{align*}\]

From before, we have:

\[\begin{split}\begin{align*} m(Z\ |\ \log\ell_{s,a})&=E[m_\ell|x_{s,a}]\\\\ &= h_\ell^2 \mathcal{N}\left(\mathbf{x}_{c,a}\ \big\vert\ \mu, W_\ell + \Sigma\right)K_{\exp(\log\ell)}(\mathbf{x}_{c,a}, \mathbf{x}_{c,a})^{-1}\bar{\ell}(\mathbf{x}_{c,a}) \end{align*}\end{split}\]

For brevity, let:

\[A_{c,a} = h_\ell^2 \mathcal{N}\left(\mathbf{x}_{c,a}\ \big\vert\ \mu, W_\ell + \Sigma\right)K_{\exp(\log\ell)}(\mathbf{x}_{c,a}, \mathbf{x}_{c,a})^{-1}\]

such that \(m(Z\ |\ \log\ell_{s,a})=A_{c,a}\bar{\ell}(\mathbf{x}_{c,a})\).

We have that \(\int\exp(cy)\mathcal{N}\left( y \ \big\vert\ m, S \right)=\exp(cm + \frac{1}{2}c^2 S)\), which allows us to marginalize out \(\bar{\ell}_a\). Thus, we need to break apart \(\bar{\ell}(\mathbf{x}_{c,a})=[\bar{\ell}_c; \bar{\ell}_a]\), giving:

\[\begin{split}\begin{align*} m(Z\ |\ \log\ell_{s,a})^2 &= \left([A_c, A_a][\bar{\ell}_c; \bar{\ell}_a]\right)^2\\\\ &= \left( A_c\bar{\ell}_c + A_a\bar{\ell}_a \right)^2\\\\ &= (A_c\bar{\ell}_c)^2 + 2(A_c\bar{\ell}_c)(A_a\bar{\ell}_a) + (A_a\bar{\ell}_a)^2 \end{align*}\end{split}\]

Plugging this into the expected squared mean:

\[\begin{split}\begin{align*} E[m_Z^2|\log\ell_{s,a}] &= \int m(Z\ |\ \log\ell_{s,a})^2 \mathcal{N}\left( \log\ell_a \ \big\vert\ \hat{m}_a, \hat{C}_a\right)\ \mathrm{d}\log\ell_a\\\\ &= (A_c\bar{\ell}_c)^2 + \int \left( 2(A_c\bar{\ell}_c)(A_a\bar{\ell}_a) + (A_a\bar{\ell}_a)^2 \right)p(\log\ell_a) \ \mathrm{d}\log\ell_a\\ &= (A_c\bar{\ell}_c)^2 + 2A_c\bar{\ell}_cA_a\int \bar{\ell}_ap(\log\ell_a)\ \mathrm{d}\log\ell_a + A_a^2\int \bar{\ell}_a^2 p(\log\ell_a) \ \mathrm{d}\log\ell_a\\ &= (A_c\bar{\ell}_c)^2 + 2A_c\bar{\ell}_cA_a\int \exp(\log\ell_a)p(\log\ell_a)\ \mathrm{d}\log\ell_a + A_a^2\int \exp(2 \log\ell_a) p(\log\ell_a) \ \mathrm{d}\log\ell_a \end{align*}\end{split}\]

Using the identity \(\int\exp(cy)\mathcal{N}\left( y \ \big\vert\ m, S \right)=\exp(cm + \frac{1}{2}c^2 S)\), we have:

\[E[m_Z^2|\log\ell_{s,a}] = A_c^2\bar{\ell}_c^2 + 2A_c\bar{\ell}_cA_a\exp\left(\hat{m}_a + \frac{1}{2}\hat{C}_a\right) + A_a^2\exp\left(2\hat{m}_a + 2\hat{C}_a\right)\]

Putting it all together, we obtain:

\[\begin{split}\begin{align*} E[V_Z|\log\ell_{s,a}] &= S(Z\ |\ \log\ell_{s}) - \left( A_c^2\bar{\ell}_c^2 + 2A_c\bar{\ell}_cA_a\exp\left(\hat{m}_a + \frac{1}{2}\hat{C}_a\right) + A_a^2\exp\left(2\hat{m}_a + 2\hat{C}_a\right)\right)\\\\ &= V_Z + m_Z^2 - \left( A_c^2\bar{\ell}_c^2 + 2A_c\bar{\ell}_cA_a\exp\left(\hat{m}_a + \frac{1}{2}\hat{C}_a\right) + A_a^2\exp\left(2\hat{m}_a + 2\hat{C}_a\right)\right) \end{align*}\end{split}\]

where \(S(Z\ |\ \log\ell_s) = V_Z + m_Z^2\) from Equation 9 of [OD12]_.

Previous topic

Bayesian Quadrature: Variance Derivation

This Page