SMC likelihood tempering?

[lukego]

Is there a convenient way to implement SMC likelihood tempering using Gen/GenParticleFilters?

I want use SMC for parameter inference on static models with complex multi-modal posteriors and broad uninformative priors. The intuition is that very few of the particles sampled from the prior will agree well with the posterior. There will need to be a series of resampling and rejuvenation steps to help the particles settle in the right places and avoid particle degeneracy along the way.

The likelihood tempering strategy is to automatically derive a series of distributions to bridge the distance between the prior and the posterior. The bridging distributions are defined by taking any black-box model and tempering its final likelihood score. Summed up well by this figure:

which shows a prior distribution (beta=0) and a posterior (beta=1) and two intermediate bridging distributions.

The challenge I see with Gen is that we don't have an explicit likelihood function that could be tempered with a wrapper function e.g. can't write

tempered_model(model, temp=0.5) = args -> model(args) ^ temp

and I'm not sure if there's an obviously correct way to hack trace.score within an inference method to affect tempering of SMC resampling/rejuvenating/reweighting steps without screwing up some important invariants on Gen data structures.

More references to the SMC likelihood tempering idea:

• Bayesian Computation Book: Sequential Monte Carlo (source of the figure.)
• SMC in PyMC.
• An Introduction to Sequential Monte Carlo section 17.2.3 (Tempering.)
• Elements of Sequential Monte Carlo end of Section 1.2.

Any hints about how to approach this problem?

I have experimented with a "white-box" approach of defining models with their own explicit internal tempering but I haven't found a satisfactory solution here. I think the automatic approach is very elegant and would like to find a way to make it work with Gen.