3. Implementation notes#

The parameters optimised in svGPFA are:

  1. variational parameters: means, \(\mathbf{m}_k^{(r)}\in\mathbb{R}^\text{n_ind_points(k)}\), and covariances, \(S_k^{(r)}\in\mathbb{R}^{\text{n_ind_points(k)}\times\text{n_ind_points(k)}}\), of the variational distributions \(q(\mathbf{u}_k^{(r)})=\mathcal{N}(\mathbf{u}_k^{(r)}|\mathbf{m}_k^{(r)}, S_k^{(r)})\) (paragraph above Eq. 4 in Duncker and Sahani [DS18]),

  2. embedding parameters: \(C\in\mathbb{R}^{\text{n_neurons}\times\text{n_latents}}\) and \(d\in\mathbb{R}^\text{n_neurons}\) (Eq. 1, middle row, in Duncker and Sahani [DS18]),

  3. kernels parameters: parameters of \(\kappa_k(\cdot,\cdot)\) in Eq. 1, top row, of Duncker and Sahani [DS18],

  4. inducing points locations: \(\mathbf{z}_k^{(r)}\in\mathbb{R}^\text{n_ind_points(k)}\) in Eq. 2 of Duncker and Sahani [DS18].

The estimation of svGPFA parameters is performed using the Expectation Conditional Maximisation algorithm (McLachlan and Krishnan [MK08], see svGPFA.stats.svEM.SVEM.maximize()), which reduces to a sequence of numerical optimisations. Because we use PyTorch’s autograd to compute derivatives, these optimisations only require the calculation of the svGPFA lower bound (left hand side of Eq. 4 in Duncker and Sahani [DS18]). Below we provide details about how the calculation of this lower bound is implemented.