Simulation from doubly-intractable distributions without perfect sampling

Abstract

In this thesis, we propose an exact auxiliary variable MCMC algorithm for doubly-intractable distributions. Such distributions arise in Bayesian inference when the likelihood has an intractable normalising constant which is a function of the likelihood parameters. The proposed algorithm is especially developed for binary MRFs, but is in principle generally applicable. We circumvent computation of the intractable normalising constant by extending the doubly-intractable distribution to include auxiliary variables such that all dependence on the intractable normalising constant is removed. This augmented distribution can then be sampled with traditional MCMC techniques. The methodology relies on the ability to construct an upper bound for the intractable normalising constant, but there is no requirement of perfect sampling from the distribution defining the likelihood. In order to apply the methodology to binary MRFs, we exploit a recursive forward-backward algorithm. We demonstrate the proposed approach via simulation examples with both a simple pairwise-interaction Ising model and a higher-order interaction MRF model, in addition to a real data example with an autologistic model. According to our experiments, the methodology performs well for fields with relatively weak interactions, but becomes too computer intensive for fields with strong interactions.