Probabilistic Classification in Business Cycles Identification Based on Generalized ROC

The area under the receiver operating characteristic (AUROC) curve is a widely used tool for assessing and ranking global classifier performance. However, because AUROC ignores the scale of predicted probabilities, it can sometimes provide a misleading performance evaluation. To address this limitation, we build on the area under the Kuipers score curve (AUKSC), and reinterpret this metric by extending the traditional ROC curve into a three‐dimensional framework that incorporates thresholds, leading to the area of the generalized ROC (AGROC) curve, thus providing a unified measure of classification performance. Through extensive Monte Carlo simulations, we demonstrate that AGROC effectively addresses the limitations of traditional AUROC metrics, offering a more robust tool for ranking probabilistic classifiers by balancing accuracy and probabilistic differentiation. In an empirical application, we show that AGROC accurately identifies recession probabilities derived from various Markov‐switching models applied to US GDP growth data, aligning closely with NBER‐defined business cycle phases.

• Publication year

  2025

• Venue

  Journal of Forecasting

• Publication date

  2025-09-03

• Fields of study

  Not labeled

• Identifiers
  DOI 10.1002/for.70020
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Real‐time weakness of the global economy

  2024cited by this paper
• ROC and PRC Approaches to Evaluate Recession Forecasts

  2023cited by this paper
• Does the Survey of Professional Forecasters Help Predict the Shape of Recessions in Real Time?

  2023cited by this paper
• Getting the ROC into Sync

  2022cited by this paper
• ROC approach to forecasting recessions using daily yield spreads

  2022cited by this paper
• Text-Based Recession Probabilities

  2022cited by this paper
• Bootstrap vs asymptotic variance estimation when using propensity score weighting with continuous and binary outcomes

  2022cited by this paper
• Tutorial overview of simple, stratified, and parametric bootstrapping

  2020cited by this paper
• Forecasting Low Frequency Macroeconomic Events with High Frequency Data

  2020cited by this paper
• Forecasting US recessions: The role of economic uncertainty☆

  2020cited by this paper
• Why Has the U.S. Economy Stagnated since the Great Recession?

  2020cited by this paper
• Markov-switching dynamic factor models in real time

  2018cited by this paper
• Forecasting the state of the Finnish business cycle

  2018cited by this paper
• Markov-Switching Models with Evolving Regime-Specific Parameters: Are Postwar Booms or Recessions All Alike?

  2016cited by this paper
• The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets

  2015cited by this paper
• A non-linear forecast combination procedure for binary outcomes

  2015cited by this paper
• The Role of Credit in Predicting US Recessions

  2015cited by this paper
• Evaluating Probability Forecasts for GDP Declines Using Alternative Methodologies

  2013cited by this paper
• Forecasting national recessions using state-level data

  2012cited by this paper
• Evaluating the Classification of Economic Activity into Recessions and Expansions

  2011cited by this paper
• Receiver operating characteristic (ROC) curve for medical researchers

  2011cited by this paper
• ROC Curves for Continuous Data

  2009cited by this paper
• AUC: a misleading measure of the performance of predictive distribution models

  2008cited by this paper
• On Bootstrapping the ROC Curve

  2008cited by this paper
• Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction

  2007cited by this paper
• Jump-and-Rest Effect of U.S. Business Cycles

  2005cited by this paper
• The ROC Curve and the Area under It as Performance Measures

  2004cited by this paper
• Evaluating collaborative filtering recommender systems

  2004cited by this paper
• Receiver operating characteristic curves and their use in radiology.

  2003cited by this paper
• A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems

  2001cited by this paper
• The use of the area under the ROC curve in the evaluation of machine learning algorithms

  1997cited by this paper
• The Application of Signal Detection Theory to Weather Forecasting Behavior

  1992cited by this paper
• A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle

  1989cited by this paper
• The Numerical Measure of the Success of Predictions

  1884cited by this paper

• No citing papers are available for this paper.