Summary: The method proposed in http://www.juhokokkala.fi/blog/posts/linear-time-evaluation-of-matern-32-gp-density-in-stan-using-the-sde-representation/ has quadratic time complexity (per evaluation of density+gradient) due to the gradient evaluation. This explains why the speedup seen in the experiment was only proportional to the number of input points (consistent with the difference between cubic and quadratic time complexity), not to the square of the number of input points as I expected. NB: I have not examined the code Stan produces, the possibility remains that the time-complexity is even higher due to something I have missed...

Read more…

Summary: Notes about implementation of a Rao-Blackwellized particle filter for approximate inference with quasiperiodic Gaussian processes and a Poisson observation model. This kind of model could be applied, for example, to predict the number customers arriving at specific time intervals, when the arrival intensity is time-varying and has daily, weekly etc. seasonality. Python implementation and an experiment with artificial data are available in an accompanying GitHub repository.

Read more…

CORRECTION 2017-06-03: The gradient evaluation here has quadratic time complexity as explained here.

Summary: Technical stuff about speeding up Bayesian inference performed with Stan, for a model with a one-dimensional latent process modeled by a Matérn 3/2 Gaussian process and a non-Gaussian likelihood (here, Poisson). The idea is that the contribution of the GP to the target density may be evaluated in linear time (w.r.t the number of observations) as opposed to the cubic time complexity of the approach based on inverting covariance matrices. (Caveat: I do not know what Stan's automatic differentation does but I expect that the time complexities are the same for the gradients. Let me know if this is false). The idea here is somewhat similar to the previous post, with the difference that this time the 'speedup' does not involve marginalizing any variables out. Instead, it only helps with a faster evaluation of exactly the same log-density (ignoring different floating-point errors). The speedup is demonstrated in an experiment, but seems to be only proportional to the number of input points (rather than to the square as one would expect based on the asymptotic complexities of only evaluating the GP density). Complete Stan codes are in an accompanying GitHub repository.

Read more…

Summary: Technical stuff about speeding up Bayesian inference performed with Stan, for a certain random-walk model that contains a conditionally linear-Gaussian state-space model. The basic idea is that we can write a Kalman-filter style loop within the Stan code so that the log-posterior evaluated is one where the state variables are marginalized out. The HMC sampler then needs to move only in the space of the parameters. Complete Stan codes are in the accompanying GitHub repository.

Read more…