A unified perspective on modified Poisson likelihoods for limited Monte Carlo data. (arXiv:1902.08831v1 [astro-ph.IM])
<a href="http://arxiv.org/find/astro-ph/1/au:+Glusenkamp_T/0/1/0/all/0/1">Thorsten Glüsenkamp</a>
Counting experiments often rely on Monte Carlo simulations for predictions of
Poisson expectations. The accompanying uncertainty from the finite Monte Carlo
sample size can be incorporated into parameter estimation by modifying the
Poisson likelihood. We first review previous Frequentist methods of this type
by Barlow et al, Bohm et al, and Chirkin, as well as recently proposed
probabilistic methods by the author and Arg”uelles et al. We show that all
these approaches can be understood in a unified way: they all approximate the
underlying probability distribution of the sum of weights in a given bin, the
compound Poisson distribution (CPD). The Probabilistic methods marginalize the
Poisson mean with a distribution that approximates the CPD, while the
Frequentist counterparts optimize the same integrand treating the mean as a
nuisance parameter. With this viewpoint we can motivate three new probabilistic
likelihoods based on generalized gamma-Poisson mixture distributions which we
derive in analytic form. Afterwards, we test old and new formulas in different
parameter estimation settings consisting of a “background” and “signal”
dataset. The new formulas outperform existing approaches in terms of
likelihood-ratio bias and coverage in all tested scenarios. We further find a
surprising outcome: usage of the exact CPD is actually bad for parameter
estimation. A continuous approximation performs much better and in principle
allows to perform bias-free inference at any level of simulated livetime if the
first two moments of the CPD of each dataset are known exactly. Finally, we
also discuss the situation where new Monte Carlo simulation is produced for a
given likelihood evaluation which leads to fluctuations in the likelihood
function. Two of the new formulas allow to include this Poisson uncertainty
directly into the likelihood which substantially decreases these fluctuations.
Counting experiments often rely on Monte Carlo simulations for predictions of
Poisson expectations. The accompanying uncertainty from the finite Monte Carlo
sample size can be incorporated into parameter estimation by modifying the
Poisson likelihood. We first review previous Frequentist methods of this type
by Barlow et al, Bohm et al, and Chirkin, as well as recently proposed
probabilistic methods by the author and Arg”uelles et al. We show that all
these approaches can be understood in a unified way: they all approximate the
underlying probability distribution of the sum of weights in a given bin, the
compound Poisson distribution (CPD). The Probabilistic methods marginalize the
Poisson mean with a distribution that approximates the CPD, while the
Frequentist counterparts optimize the same integrand treating the mean as a
nuisance parameter. With this viewpoint we can motivate three new probabilistic
likelihoods based on generalized gamma-Poisson mixture distributions which we
derive in analytic form. Afterwards, we test old and new formulas in different
parameter estimation settings consisting of a “background” and “signal”
dataset. The new formulas outperform existing approaches in terms of
likelihood-ratio bias and coverage in all tested scenarios. We further find a
surprising outcome: usage of the exact CPD is actually bad for parameter
estimation. A continuous approximation performs much better and in principle
allows to perform bias-free inference at any level of simulated livetime if the
first two moments of the CPD of each dataset are known exactly. Finally, we
also discuss the situation where new Monte Carlo simulation is produced for a
given likelihood evaluation which leads to fluctuations in the likelihood
function. Two of the new formulas allow to include this Poisson uncertainty
directly into the likelihood which substantially decreases these fluctuations.
http://arxiv.org/icons/sfx.gif
