Evaluation of probabilistic photometric redshift estimation approaches for LSST. (arXiv:2001.03621v1 [astro-ph.CO])
<a href="http://arxiv.org/find/astro-ph/1/au:+Schmidt_S/0/1/0/all/0/1">S.J. Schmidt</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Malz_A/0/1/0/all/0/1">A.I. Malz</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Soo_J/0/1/0/all/0/1">J.Y.H. Soo</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Almosallam_I/0/1/0/all/0/1">I.A. Almosallam</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Brescia_M/0/1/0/all/0/1">M. Brescia</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Cavuoti_S/0/1/0/all/0/1">S. Cavuoti</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Cohen_Tanugi_J/0/1/0/all/0/1">J. Cohen-Tanugi</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Connolly_A/0/1/0/all/0/1">A.J. Connolly</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+DeRose_J/0/1/0/all/0/1">J. DeRose</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Freeman_P/0/1/0/all/0/1">P.E. Freeman</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Graham_M/0/1/0/all/0/1">M.L. Graham</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Iyer_K/0/1/0/all/0/1">K.G. Iyer</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Jarvis_M/0/1/0/all/0/1">M.J. Jarvis</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Kalmbach_J/0/1/0/all/0/1">J.B. Kalmbach</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Kovacs_E/0/1/0/all/0/1">E. Kovacs</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Lee_A/0/1/0/all/0/1">A.B. Lee</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Longo_G/0/1/0/all/0/1">G. Longo</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Morrison_C/0/1/0/all/0/1">C.B. Morrison</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Newman_J/0/1/0/all/0/1">J.A. Newman</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Nourbakhsh_E/0/1/0/all/0/1">E. Nourbakhsh</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Nuss_E/0/1/0/all/0/1">E. Nuss</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Pospisil_T/0/1/0/all/0/1">T. Pospisil</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Tranin_H/0/1/0/all/0/1">H. Tranin</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Wechsler_R/0/1/0/all/0/1">R.H. Wechsler</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Zhou_R/0/1/0/all/0/1">R. Zhou</a>, <a href="http://arxiv.org/find/astro-ph/1/au:+Izbicki_R/0/1/0/all/0/1">R. Izbicki</a>, The <a href="http://arxiv.org/find/astro-ph/1/au:+Collaboration_LSST_Dark_Energy_Science/0/1/0/all/0/1">LSST Dark Energy Science Collaboration</a>
Many scientific investigations of photometric galaxy surveys require redshift
estimates, whose uncertainty properties are best encapsulated by photometric
redshift (photo-z) posterior probability density functions (PDFs). A plethora
of photo-z PDF estimation methodologies abound, producing discrepant results
with no consensus on a preferred approach. We present the results of a
comprehensive experiment comparing twelve photo-z algorithms applied to mock
data produced for the Large Synoptic Survey Telescope (LSST) Dark Energy
Science Collaboration (DESC). By supplying perfect prior information, in the
form of the complete template library and a representative training set as
inputs to each code, we demonstrate the impact of the assumptions underlying
each technique on the output photo-z PDFs. In the absence of a notion of true,
unbiased photo-z PDFs, we evaluate and interpret multiple metrics of the
ensemble properties of the derived photo-z PDFs as well as traditional
reductions to photo-z point estimates. We report systematic biases and overall
over/under-breadth of the photo-z PDFs of many popular codes, which may
indicate avenues for improvement in the algorithms or implementations.
Furthermore, we raise attention to the limitations of established metrics for
assessing photo-z PDF accuracy; though we identify the conditional density
estimate (CDE) loss as a promising metric of photo-z PDF performance in the
case where true redshifts are available but true photo-z PDFs are not, we
emphasize the need for science-specific performance metrics.
Many scientific investigations of photometric galaxy surveys require redshift
estimates, whose uncertainty properties are best encapsulated by photometric
redshift (photo-z) posterior probability density functions (PDFs). A plethora
of photo-z PDF estimation methodologies abound, producing discrepant results
with no consensus on a preferred approach. We present the results of a
comprehensive experiment comparing twelve photo-z algorithms applied to mock
data produced for the Large Synoptic Survey Telescope (LSST) Dark Energy
Science Collaboration (DESC). By supplying perfect prior information, in the
form of the complete template library and a representative training set as
inputs to each code, we demonstrate the impact of the assumptions underlying
each technique on the output photo-z PDFs. In the absence of a notion of true,
unbiased photo-z PDFs, we evaluate and interpret multiple metrics of the
ensemble properties of the derived photo-z PDFs as well as traditional
reductions to photo-z point estimates. We report systematic biases and overall
over/under-breadth of the photo-z PDFs of many popular codes, which may
indicate avenues for improvement in the algorithms or implementations.
Furthermore, we raise attention to the limitations of established metrics for
assessing photo-z PDF accuracy; though we identify the conditional density
estimate (CDE) loss as a promising metric of photo-z PDF performance in the
case where true redshifts are available but true photo-z PDFs are not, we
emphasize the need for science-specific performance metrics.
http://arxiv.org/icons/sfx.gif
