http://www.juhokokkala.fi/blog/enSat, 03 Jun 2017 14:43:08 GMThttps://getnikola.com/http://blogs.law.harvard.edu/tech/rsshttp://www.juhokokkala.fi/blog/posts/correction-the-evaluation-of-matern-32-gp-density-takes-quadratic-time/Juho Kokkala<div><p>Summary: The method proposed in <a class="reference external" href="http://www.juhokokkala.fi/blog/posts/linear-time-evaluation-of-matern-32-gp-density-in-stan-using-the-sde-representation/">http://www.juhokokkala.fi/blog/posts/linear-time-evaluation-of-matern-32-gp-density-in-stan-using-the-sde-representation/</a> has quadratic time complexity (per evaluation of density+gradient) due to the gradient evaluation. This explains why the speedup seen in the experiment was only proportional to the number of input points (consistent with the difference between cubic and quadratic time complexity), not to the square of the number of input points as I expected. NB: I have not examined the code Stan produces, the possibility remains that the time-complexity is even higher due to something I have missed...</p> <p><a href="http://www.juhokokkala.fi/blog/posts/correction-the-evaluation-of-matern-32-gp-density-takes-quadratic-time/">Read more…</a> (1 min remaining to read)</p></div>bayescorrectionsgaussian-processkalman-filtermathjaxsdestanhttp://www.juhokokkala.fi/blog/posts/correction-the-evaluation-of-matern-32-gp-density-takes-quadratic-time/Sat, 03 Jun 2017 14:42:00 GMThttp://www.juhokokkala.fi/blog/posts/rao-blackwellized-particle-filters-for-quasiperiodic-gaussian-processes/Juho Kokkala<div><p>Summary: Notes about implementation of a Rao-Blackwellized particle filter for approximate inference with quasiperiodic Gaussian processes and a Poisson observation model. This kind of model could be applied, for example, to predict the number customers arriving at specific time intervals, when the arrival intensity is time-varying and has daily, weekly etc. seasonality. Python implementation and an experiment with artificial data are available in an <a class="reference external" href="http://github.com/juhokokkala/rbpf_qpgp">accompanying GitHub repository</a>.</p> <p><a href="http://www.juhokokkala.fi/blog/posts/rao-blackwellized-particle-filters-for-quasiperiodic-gaussian-processes/">Read more…</a> (5 min remaining to read)</p></div>bayesgaussian-processGPykalman-filtermathjaxparticle-filterhttp://www.juhokokkala.fi/blog/posts/rao-blackwellized-particle-filters-for-quasiperiodic-gaussian-processes/Wed, 31 Aug 2016 09:48:00 GMThttp://www.juhokokkala.fi/blog/posts/linear-time-evaluation-of-matern-32-gp-density-in-stan-using-the-sde-representation/Juho Kokkala<div><p>CORRECTION 2017-06-03: The gradient evaluation here has quadratic time complexity <a class="reference external" href="http://www.juhokokkala.fi/blog/posts/correction-the-evaluation-of-matern-32-gp-density-takes-quadratic-time/">as explained here</a>.</p> <p>Summary: Technical stuff about speeding up Bayesian inference performed with <a class="reference external" href="http://mc-stan.org">Stan</a>, for a model with a one-dimensional latent process modeled by a Matérn 3/2 Gaussian process and a non-Gaussian likelihood (here, Poisson). The idea is that the contribution of the GP to the target density may be evaluated in linear time (w.r.t the number of observations) as opposed to the cubic time complexity of the approach based on inverting covariance matrices. (Caveat: I do not know what Stan's automatic differentation does but I expect that the time complexities are the same for the gradients. Let me know if this is false). The idea here is somewhat similar to the previous post, with the difference that this time the 'speedup' does not involve marginalizing any variables out. Instead, it only helps with a faster evaluation of exactly the same log-density (ignoring different floating-point errors). The speedup is demonstrated in an experiment, but seems to be only proportional to the number of input points (rather than to the square as one would expect based on the asymptotic complexities of only evaluating the GP density). Complete Stan codes are in an <a class="reference external" href="http://github.com/juhokokkala/matern32-poisson-sde-stan">accompanying GitHub repository</a>.</p> <p><a href="http://www.juhokokkala.fi/blog/posts/linear-time-evaluation-of-matern-32-gp-density-in-stan-using-the-sde-representation/">Read more…</a> (18 min remaining to read)</p></div>bayesgaussian-processkalman-filtermathjaxsdestanhttp://www.juhokokkala.fi/blog/posts/linear-time-evaluation-of-matern-32-gp-density-in-stan-using-the-sde-representation/Sat, 13 Aug 2016 17:04:00 GMThttp://www.juhokokkala.fi/blog/posts/kalman-filter-style-recursion-to-marginalize-state-variables-to-speed-up-stan-inference/Juho Kokkala<div><p>Summary: Technical stuff about speeding up Bayesian inference performed with <a class="reference external" href="http://mc-stan.org">Stan</a>, for a certain random-walk model that contains a conditionally linear-Gaussian state-space model. The basic idea is that we can write a Kalman-filter style loop within the Stan code so that the log-posterior evaluated is one where the state variables are marginalized out. The HMC sampler then needs to move only in the space of the parameters. Complete Stan codes are in the accompanying <a class="reference external" href="https://github.com/juhokokkala/kalman-stan-randomwalk">GitHub repository</a>.</p> <p><a href="http://www.juhokokkala.fi/blog/posts/kalman-filter-style-recursion-to-marginalize-state-variables-to-speed-up-stan-inference/">Read more…</a> (7 min remaining to read)</p></div>bayeskalman-filtermathjaxstanhttp://www.juhokokkala.fi/blog/posts/kalman-filter-style-recursion-to-marginalize-state-variables-to-speed-up-stan-inference/Mon, 08 Aug 2016 15:15:00 GMThttp://www.juhokokkala.fi/blog/posts/hello-world/Juho Kokkala<p>This is a test post.</p>http://www.juhokokkala.fi/blog/posts/hello-world/Mon, 08 Aug 2016 08:29:00 GMT