SCIENCE@home

Calibration training for improving probabilistic judgments using an interactive app

Ross Gruetzemacher,Kang Bok Lee,David Paradice

Published 2023 in FUTURES & FORESIGHT SCIENCE

ABSTRACT

We describe an exploratory study examining the effectiveness of an interactive app and a novel training process for improving calibration and reducing overconfidence in probabilistic judgments. We evaluated the training used in the app by conducting an American college football forecasting tournament involving 153 business school students making 52 forecasts over 11 weeks. A coarsened exact matching analysis found statistical evidence that, in under 30 min, the more challenging training was able to modestly reduce overconfidence, improve calibration and improve the accuracy of probabilistic judgments (measured by the Brier score). The experimental results also suggest that the generic training can generalize across domains and that effective calibration training is possible without expert facilitators or pedagogical training materials. Although no previous studies have reported similar results, due to the modest effect, we conclude that these results should only be interpreted as a proof of concept and that further evaluation and validation of mechanisms of the app's effect is necessary.

PUBLICATION RECORD

  • Publication year

    2023

  • Venue

    FUTURES & FORESIGHT SCIENCE

  • Publication date

    2023-12-28

  • Fields of study

    Not labeled

  • Identifiers
    DOI 10.1002/ffo2.177
  • External record

    Open on Semantic Scholar

  • Source metadata

    Semantic Scholar

CITATION MAP

EXTRACTION MAP

CLAIMS

  • No claims are published for this paper.

CONCEPTS

  • No concepts are published for this paper.

REFERENCES

Showing 1-37 of 37 references · Page 1 of 1

CITED BY