Why do some probabilistic forecasts lack reliability?. (arXiv:1904.08791v1 [physics.space-ph] CROSS LISTED)
<a href="http://arxiv.org/find/physics/1/au:+Kubo_Y/0/1/0/all/0/1">Y&#xfb;ki Kubo</a>

In this work, we investigate the reliability of the probabilistic binary
forecast. We mathematically prove that a necessary, but not sufficient,
condition for achieving a reliable probabilistic forecast is maximizing the
Peirce skill score (PSS) at the threshold probability of the climatological
base rate. The condition is confirmed by using artificially synthesized
forecast-outcome pair data and previously published probabilistic solar flare
forecast models. The condition gives a partial answer as to why some
probabilistic forecast system lack reliability, because the system, which does
not satisfy the proved condition, can never be reliable. Therefore, the proved
condition is very important for the developers of a probabilistic forecast
system. The result implies that those who want to develop a reliable
probabilistic forecast system must adjust or train the system so as to maximize
PSS near the threshold probability of the climatological base rate.

In this work, we investigate the reliability of the probabilistic binary
forecast. We mathematically prove that a necessary, but not sufficient,
condition for achieving a reliable probabilistic forecast is maximizing the
Peirce skill score (PSS) at the threshold probability of the climatological
base rate. The condition is confirmed by using artificially synthesized
forecast-outcome pair data and previously published probabilistic solar flare
forecast models. The condition gives a partial answer as to why some
probabilistic forecast system lack reliability, because the system, which does
not satisfy the proved condition, can never be reliable. Therefore, the proved
condition is very important for the developers of a probabilistic forecast
system. The result implies that those who want to develop a reliable
probabilistic forecast system must adjust or train the system so as to maximize
PSS near the threshold probability of the climatological base rate.

http://arxiv.org/icons/sfx.gif