Comparison of unfolding methods using RooFitUnfold
- Brenner, Lydia et al
- arXiv:1910.14654
 | Left: The true distributions for the SM and BSM models and the corresponding smeared datasets. Right: the transfer matrix for the SM model, which is populated by the same events as the SM true distribution shown. |
 | Bias for different regularisation depths for Iterative Bayes. Regularisation of 1 (left) and 4 (right) are shown on the first line and regularisation of 100 (left) and 500 (right) on the second line. |
 | Results at truth level for different regularisation depths for Iterative Bayes. Regularisation of 1 (left) and 4 (right) are shown on the first line and regularisation of 100 (left) and 500 (right) on the second line. |
 | Left: The true distributions for the SM and BSM models and the corresponding smeared datasets. Right: the transfer matrix for the SM model, which is populated by the same events as the SM true distribution shown. |
 | Results at truth level for different regularisation depths for Iterative Bayes. Regularisation of 1 (left) and 4 (right) are shown on the first line and regularisation of 100 (left) and 500 (right) on the second line. |
 | : Comparison of the unfolding performance for the algorithms discussed in this paper. For each method, the top plot shows the unfolded SM data compared to the SM true distribution, where SM response matrix was used as input to the unfolding method. For the methods with a tuneable regularisation strength two unfolding solutions are shown: (1) with a regularisation strength corresponding to the unconditionally minimised MSE (red), and (2) with a conditionally minimised MSE with the requirement that the bin-averaged coverage reaches the target coverage within 1\% (grey). The middle and lower panels of the top row plots display the bin-bin coverage, variance and bias estimates. For the bin-by-bin bias, the uncertainty of the toy-based estimate is shown with an error bar. Also overlaid on the bias plot is the statistical error of the unfolded data (i.e the RMS of the distribution of $\hat{\mu}$) to help guide the interpretation of observed fluctuations in the unfolded data. For the tuneable methods, the bin-averaged MSE and the bin-averaged coverage, which are used to tune regularisation strength, are shown as function of that strength in the bottom pane. The vertical lines in the bottom pane indicate the regularisation strength solutions chosen by the optimisation methods (1) and (2). |
 | : Comparison of the unfolding performance for the algorithms discussed in this paper. For each method, the top plot shows the unfolded BSM data compared to the BSM true distribution, where {\em SM }response matrix was used as input to the unfolding method. For the methods with a tuneable regularisation strength two unfolding solutions are shown: (1) with a regularisation strength corresponding to the unconditionally minimised MSE (red), and (2) with a conditionally minimised MSE with the requirement that the bin-averaged coverage reaches the target coverage within 1\% (grey). In this result some of the unfolded data point fall outside the plot range, and are therefore not shown. The middle and lower panels of the top row plots display the bin-bin coverage, variance and bias estimates. For the bin-by-bin bias, the uncertainty of the toy-based estimate is shown with an error bar. Also overlaid on the bias plot is the statistical error of the unfolded data (i.e the RMS of the distribution of $\hat{\mu}$) to help guide the interpretation of observed fluctuations in the unfolded data. For the tuneable methods, the bin-averaged MSE and the bin-averaged coverage, which are used to tune regularisation strength, are shown as function of that strength in the bottom pane. The vertical lines in the bottom pane indicate the regularisation strength solutions chosen by the optimisation methods (1) and (2). |
 | Reconstruction level comparison between the data and the theory (left) The response matrix (right) |
 | : Caption not extracted |
 | Bias of the different methods. On the first line Iterative Bayes (left) and Gaussian Processes (right). On the second line IDS (left) and SVD (right). On the third line Bin-By-Bin (left) and Matrix inversion (right) and on the last line TUnfold. |
 | : Caption not extracted |
 | Bias of the different methods. On the first line Iterative Bayes (left) and Gaussian Processes (right). On the second line IDS (left) and SVD (right). On the third line Bin-By-Bin (left) and Matrix inversion (right) and on the last line TUnfold. |
 | : Caption not extracted |
 | Results at truth level for different methods. On the first line Iterative Bayes with a regularisation depth of 4 (left) and Gaussian Processes where the regularisation is found by maximising a marginal likelihood (right). On the second line IDS with a regularisation depth of 4 (left) and SVD with a regularisation depth of 11 (right). On the third line Bin-By-Bin (left) and Matrix inversion (right) and on the last line TUnfold with a regularisation depth of 0.01. |
 | : Caption not extracted |
 | Results at truth level for different methods. On the first line Iterative Bayes with a regularisation depth of 4 (left) and Gaussian Processes where the regularisation is found by maximising a marginal likelihood (right). On the second line IDS with a regularisation depth of 4 (left) and SVD with a regularisation depth of 11 (right). On the third line Bin-By-Bin (left) and Matrix inversion (right) and on the last line TUnfold with a regularisation depth of 0.01. |
 | : Caption not extracted |
 | Results at truth level for different methods. On the first line Iterative Bayes with a regularisation depth of 4 (left) and Gaussian Processes where the regularisation is found by maximising a marginal likelihood (right). On the second line IDS with a regularisation depth of 4 (left) and SVD with a regularisation depth of 11 (right). On the third line Bin-By-Bin (left) and Matrix inversion (right) and on the last line TUnfold with a regularisation depth of 0.01. |
 | Bias of the different methods. On the first line Iterative Bayes (left) and Gaussian Processes (right). On the second line IDS (left) and SVD (right). On the third line Bin-By-Bin (left) and Matrix inversion (right) and on the last line TUnfold. |
 | : Caption not extracted |
 | : Caption not extracted |
 | Bias for different regularisation depths for Iterative Bayes. Regularisation of 1 (left) and 4 (right) are shown on the first line and regularisation of 100 (left) and 500 (right) on the second line. |
 | : Caption not extracted |
 | Bias for different regularisation depths for Iterative Bayes. Regularisation of 1 (left) and 4 (right) are shown on the first line and regularisation of 100 (left) and 500 (right) on the second line. |
 | : Caption not extracted |
 | Reconstruction level comparison between the data and the theory (left) The response matrix (right) |
 | : Caption not extracted |
 | Reconstruction level comparison between the data and the theory (left) The response matrix (right) |
 | : Caption not extracted |
 | Bias of the different methods. On the first line Iterative Bayes (left) and Gaussian Processes (right). On the second line IDS (left) and SVD (right). On the third line Bin-By-Bin (left) and Matrix inversion (right) and on the last line TUnfold. |
 | : Caption not extracted |
 | Bias of the different methods. On the first line Iterative Bayes (left) and Gaussian Processes (right). On the second line IDS (left) and SVD (right). On the third line Bin-By-Bin (left) and Matrix inversion (right) and on the last line TUnfold. |
 | Bias of the different methods. On the first line Iterative Bayes (left) and Gaussian Processes (right). On the second line IDS (left) and SVD (right). On the third line Bin-By-Bin (left) and Matrix inversion (right) and on the last line TUnfold. |
 | : Caption not extracted |
 | Results at truth level for different methods. On the first line Iterative Bayes with a regularisation depth of 4 (left) and Gaussian Processes where the regularisation is found by maximising a marginal likelihood (right). On the second line IDS with a regularisation depth of 4 (left) and SVD with a regularisation depth of 11 (right). On the third line Bin-By-Bin (left) and Matrix inversion (right) and on the last line TUnfold with a regularisation depth of 0.01. |
 | : Caption not extracted |
 | Results at truth level for the Bin-By-Bin method (left) and its corresponding bias (right) for the di-photon example. |
 | Results at truth level for the Bin-By-Bin method (left) and its corresponding bias (right) for the di-photon example. |
 | : Caption not extracted |
 | : Caption not extracted |
 | Reconstruction level comparison between the data and the theory (left) The response matrix (right) |
 | : Caption not extracted |
 | Results at truth level for different methods. On the first line Iterative Bayes with a regularisation depth of 4 (left) and Gaussian Processes where the regularisation is found by maximising a marginal likelihood (right). On the second line IDS with a regularisation depth of 4 (left) and SVD with a regularisation depth of 11 (right). On the third line Bin-By-Bin (left) and Matrix inversion (right) and on the last line TUnfold with a regularisation depth of 0.01. |
 | : Caption not extracted |
 | Results at truth level for different regularisation depths for Iterative Bayes. Regularisation of 1 (left) and 4 (right) are shown on the first line and regularisation of 100 (left) and 500 (right) on the second line. |
 | Visualization of the true distribution of the bimodal model for various values of the distortion parameter $\alpha$. |
 | Bias of the different methods. On the first line Iterative Bayes (left) and Gaussian Processes (right). On the second line IDS (left) and SVD (right). On the third line Bin-By-Bin (left) and Matrix inversion (right) and on the last line TUnfold. |
 | Reconstruction level comparison between the data and the theory (left) The response matrix (right) |
 | Visualization of the true distribution of the bimodal model for various values of the distortion parameter $\alpha$. |
 | Bias of the different methods. On the first line Iterative Bayes (left) and Gaussian Processes (right). On the second line IDS (left) and SVD (right). On the third line Bin-By-Bin (left) and Matrix inversion (right) and on the last line TUnfold. |
 | Results at truth level for different methods. On the first line Iterative Bayes with a regularisation depth of 4 (left) and Gaussian Processes where the regularisation is found by maximising a marginal likelihood (right). On the second line IDS with a regularisation depth of 4 (left) and SVD with a regularisation depth of 11 (right). On the third line Bin-By-Bin (left) and Matrix inversion (right) and on the last line TUnfold with a regularisation depth of 0.01. |
 | Visualization of the true distribution of the bimodal model for various values of the distortion parameter $\alpha$. |
 | Results at truth level for different methods. On the first line Iterative Bayes with a regularisation depth of 4 (left) and Gaussian Processes where the regularisation is found by maximising a marginal likelihood (right). On the second line IDS with a regularisation depth of 4 (left) and SVD with a regularisation depth of 11 (right). On the third line Bin-By-Bin (left) and Matrix inversion (right) and on the last line TUnfold with a regularisation depth of 0.01. |
 | Visualization of the true distribution of the bimodal model for various values of the distortion parameter $\alpha$. |
 | Visualization of the true distribution of the bimodal model for various values of the distortion parameter $\alpha$. |
 | Results at truth level for different regularisation depths for Iterative Bayes. Regularisation of 1 (left) and 4 (right) are shown on the first line and regularisation of 100 (left) and 500 (right) on the second line. |
 | Bias for different regularisation depths for Iterative Bayes. Regularisation of 1 (left) and 4 (right) are shown on the first line and regularisation of 100 (left) and 500 (right) on the second line. |
 | Visualization of the true distribution of the bimodal model for various values of the distortion parameter $\alpha$. |
 | Comparison of the estimated bin-averaged bias of the tuneable unfolding algorithms when unfolding the distorted bimodal model with a response matrix obtained from the undistorted model. Results are shown for various distortion strengths, which increase in distortion with decreasing values of $\alpha$ (See Fig.~\protect{\ref{fig:scan_bimodal}}). In this study, the regularisation strength for each algorithm has been unconditionally optimised on the MSE. |
 | Comparison of the estimated bin-averaged bias of the tuneable unfolding algorithms when unfolding the distorted bimodal model with a response matrix obtained from the undistorted model. Results are shown for various distortion strengths, which increase in distortion with decreasing values of $\alpha$ (See Fig.~\protect{\ref{fig:scan_bimodal}}). In this study, the regularisation strength for each algorithm has been optimised on the MSE with the condition that coverage is achieved with 1\% of the target value |
 | Comparison of the estimated bin-averaged bias of non-tuneable and non-regularised methods when unfolding the distorted bimodal model with a response matrix obtained from the undistorted model. Results are shown for various distortion strengths, which increase in distortion with decreasing values of $\alpha$ (See Fig.~\protect{\ref{fig:scan_bimodal}}). |