Expert Status and Performance

Expert judgements are essential when time and resources are stretched or we face novel dilemmas requiring fast solutions. Good advice can save lives and large sums of money. Typically, experts are defined by their qualifications, track record and experience [1], [2]. The social expectation hypothesis argues that more highly regarded and more experienced experts will give better advice. We asked experts to predict how they will perform, and how their peers will perform, on sets of questions. The results indicate that the way experts regard each other is consistent, but unfortunately, ranks are a poor guide to actual performance. Expert advice will be more accurate if technical decisions routinely use broadly-defined expert groups, structured question protocols and feedback.

• Publication year

  2011

• Venue

  PLoS ONE

• Publication date

  2011-07-29

• Fields of study

  Medicine, Economics, Psychology

• Identifiers
  DOI 10.1371/journal.pone.0022998PMID 21829574PMCID 3146531
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Reducing Overconfidence in the Interval Judgments of Experts

  2010cited by this paper
• A route to more tractable expert advice

  2010cited by this paper
• The Perfect Swarm: The Science of Complexity in Everyday Life

  2009cited by this paper
• Rethinking Expertise

  2008cited by this paper
• When Should Consumers and Managers Trust Their Intuition

  2006cited by this paper
• The Cambridge handbook of expertise and expert performance

  2006cited by this paper
• The Cambridge handbook of expertise and expert performance

  2006cited by this paper
• Two Approaches to the Study of Experts' Characteristics

  2006cited by this paper
• Another look at measures of forecast accuracy

  2006cited by this paper
• Contracting: A New Form of Professional Practice

  2006cited by this paper
• Statistical Methods for Eliciting Probability Distributions

  2005cited by this paper
• Risks and Decisions for Conservation and Environmental Management: Risk ranking

  2005cited by this paper
• Probability is perfect, but we can't elicit it perfectly

  2004cited by this paper
• Elicitation of expert opinions for uncertainty and risks

  2003cited by this paper
• Representation facilitates reasoning: what natural frequencies are and what they are not.

  2002cited by this paper
• Procedures Guide for Structural Expert Judgement in Accident Consequence Modelling

  2000cited by this paper
• Trust, Emotion, Sex, Politics, and Science: Surveying the Risk‐Assessment Battlefield

  1999cited by this paper
• The moderating effects of problem characteristics on experts' and novices' judgments

  1997cited by this paper
• Appeal to Expert Opinion: Arguments from Authority

  1997cited by this paper
• Research on Judgment and Decision Making: Currents, Connections, and Controversies

  1997cited by this paper
• What Is a Good Forecast? An Essay on the Nature of Goodness in Weather Forecasting

  1993cited by this paper
• Acquisition of Expert Judgment: Examples from Risk Assessment

  1992cited by this paper
• Experts in Uncertainty: Opinion and Subjective Probability in Science

  1991influential reference
• The process-performance paradox in expert judgment - How can experts know so much and predict so badly?

  1991cited by this paper
• Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis

  1990cited by this paper
• Knowledge acquisition for expert systems

  1988cited by this paper
• Judgment Under Uncertainty: Heuristics and Biases.

  1984cited by this paper
• The Comparison and Evaluation of Forecasters.

  1983cited by this paper
• Lay Foibles and Expert Fables in Judgments about Risk

  1982cited by this paper
• Judgment under uncertainty: Frontmatter

  1982cited by this paper
• Interactive Elicitation of Opinion for a Normal Linear Model

  1980cited by this paper
• CAN WEATHER FORECASTERS FORMULATE RELIABLE PROBABILITY FORECASTS OF PRECIPITATION AND TEMPERATURES ?

  1977cited by this paper
• Elicitation of Personal Probabilities and Expectations

  1971cited by this paper
• The Quantification of Judgment: Some Methodological Suggestions

  1967cited by this paper
• The Scientist Speculates: An Anthology of Partly-baked Ideas

  1964cited by this paper

• An expert elicitation approach for characterizing marine mammal and small-scale fisheries interactions: Insights from Chilean dolphin (Cephalorhynchus eutropia) bycatch

  2026cites this paper
• Expert elicitation study for sustainable small hydropower (SSHP) planning using MCDA techniques

  2026cites this paper
• A Network Approach to Support Sustainability Policy Coordination: Exploring Linkages Between Biodiversity Indicators and Essential Biodiversity Variables

  2026cites this paper
• Constructed value of information with iterative scoring and parametric uncertainty to identify management-relevant research priorities for a declining raptor species.

  2026cites this paper
• Using constructed value of information to identify key uncertainties for a decision tree analysis in iterative structured decision making

  2026cites this paper
• Testing and improving the usefulness of invertebrate indicators of multiple freshwater stressors

  2025cites this paper
• Investigating the current implementation barriers and the potential social and ecological effects of a CITES reverse listing on the international exotic pet trade

  2025cites this paper
• Combining Statistical Evidence When Evidence Is Measured by Relative Belief

  2025cites this paper
• Using constructed value of information to evaluate research needs in conservation strategy assumptions

  2025cites this paper
• A place-based assessment of biodiversity intactness in sub-Saharan Africa

  2025influential citation
• Mitigation of human cognitive bias in volcanic eruption forecasting

  2025influential citation
• Using expert elicitation to assess the likely effectiveness of conservation interventions during an unprecedented outbreak of high pathogenicity avian influenza (HPAI) in wild birds

  2025cites this paper
• Assessing recovery and conservation of Australian freshwater fishes with the IUCN Green Status of Species and structured expert elicitation

  2025cites this paper
• Review of evidence that foxes and cats cause extinctions of Australia's endemic mammals

  2025cites this paper
• A novel method for plant species conservation prioritisation at a local scale: the IDEA protocol and beyond

  2025cites this paper
• Estimating the costs of interrelated reproductive disorders in dairy farms.

  2025cites this paper
• STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reports

  2025cites this paper
• Stakeholder Engagement: An Expertise-Centred Approach

  2025cites this paper
• Do We Practice What We Preach? A Mixed Methods Study of Stress in Stress Experts: Implications for Transfer of Awareness and Learning

  2025cites this paper
• A Subjective and Intuitive Approach to Rapid, Holistic Assessment of Natural Ecosystem Integrity Across a Community‐Managed Conservation Area in Southern Tanzania

  2025cites this paper
• A practical framework to evaluate the feasibility of incentive-based approaches to reduce bycatch of marine mammals and other protected species

  2025cites this paper
• Behavioural response thresholds for the assessment of noise impact on Antarctic marine mammal species

  2025cites this paper
• Feral cats and habitat degradation drive mammal declines in northwestern Australia: An expert elicitation to prioritize management actions

  2025cites this paper
• Small decisions as social and ecological leverage points for cities to meet sustainability goals: a synthesis for urban forestry research and practice

  2025cites this paper
• Main parasitic infections in gilthead seabream and European seabass aquaculture: risk factors from stakeholders’ perspective

  2024cites this paper
• Balancing observational data and experiential knowledge in environmental flows modeling

  2024cites this paper
• A practical stakeholder-focused approach for assessing the biosecurity system in Pacific Island countries

  2024cites this paper
• Evaluation of management options for climate-change adaptation of threatened species: a case study of a restricted orchid

  2024cites this paper
• Comparative Evaluation of Multi‐Criteria Decision Making Methods for River Hydromorphology Assessment: Case Study of the Saz‐Çayırova Basin, Türkiye

  2024cites this paper
• Quantitative support for the benefits of proactive management for wildlife disease control

  2024cites this paper
• Internal communication from a happiness management perspective: state-of-the-art and theoretical construction of a guide for its development

  2024cites this paper
• Risk‐based valuation of surveillance data in open environments: Methods application to a key shellfish aquaculture production region

  2024cites this paper
• Parameter Estimation of a Valve-Controlled Cylinder System Model Based on Bench Test and Operating Data Fusion

  2024cites this paper
• Assessing invasion risks using EICAT-based expert elicitation: application to a conservation translocation

  2024cites this paper
• Assessing coverage of the monitoring framework of the Kunming-Montreal Global Biodiversity Framework and opportunities to fill gaps

  2024cites this paper
• Experts’ Perspectives on Inclusive Governance for Protecting Hot Spring Landscapes in China: Barriers and Implications

  2024cites this paper
• Cumulative effects of multiple stressors impact an endangered seagrass population and fish communities.

  2023cites this paper
• How expert insight into alpine peatland conservation complements global scientific evidence

  2023cites this paper
• The role of data‐based intelligence and experience on time efficiency of taxi drivers: An empirical investigation using large‐scale sensor data

  2023cites this paper
• Vulnerability assessment of the pink shrimp small-scale fishery to climate change in southwestern Atlantic brackish coastal lagoons (Uruguay)

  2023cites this paper
• A methodology for using a multilevel perspective framework to analyze complex systems

  2023cites this paper
• Qualitative value of information provides a transparent and repeatable method for identifying critical uncertainty.

  2023cites this paper
• Bones of contention: a double-blind study of experts’ ability to classify sheep and goat astragali from images

  2023cites this paper
• How expert are ‘experts’? Comparing expert predictions and empirical data on the use of farmland restoration sites by birds

  2023cites this paper
• Assessing parent interaction with deaf infants: A quantitative survey of UK professional practice.

  2023cites this paper
• Expert Judgment in the 2022 Aotearoa New Zealand National Seismic Hazard Model

  2023cites this paper
• Validating a forced-choice method for eliciting quality-of-reasoning judgments

  2023cites this paper
• A Bayesian Hierarchical Model of Crowd Wisdom Based on Predicting Opinions of Others

  2023cites this paper
• The value of capturing diverse perspectives when setting decision triggers for threatened species management

  2023cites this paper
• A new methodology to extrapolate disease freedom to an area using surveillance results from selected aquatic populations.

  2023cites this paper
• Development and Evaluation of Options for Action to Progress on the SDG 6 Targets in Austria.

  2023cites this paper
• Co-designing a multi-criteria approach to ranking hazards to and from Australia’s emerging offshore blue economy

  2023cites this paper
• IDEAcology: An interface to streamline and facilitate efficient, rigorous expert elicitation in ecology

  2023cites this paper
• A synthesis of approaches to support integrated assessments of hazards for the emerging Blue Economy

  2023cites this paper
• Animal population decline and recovery after severe fire: Relating ecological and life history traits with expert estimates of population impacts from the Australian 2019-20 megafires

  2023cites this paper
• Comparing wildlife habitat suitability models based on expert opinion with camera trap detections

  2023cites this paper
• The importance of expert selection when identifying threatened ecosystems

  2023cites this paper
• Establishing effective conservation management strategies for a poorly known endangered species: a case study using Australia’s Night Parrot (Pezoporus occidentalis)

  2023influential citation
• Informative Priors for the Consensus Ranking in the Bayesian Mallows Model

  2022cites this paper
• Linguistic capital and status: The interaction between language skills, personal reputation, and perceived collaboration performance

  2022cites this paper
• Crowdsourced Perceptions of Human Behavior to Improve Computational Forecasts of US National Incident Cases of COVID-19: Survey Study

  2022cites this paper
• Reimagining peer review as an expert elicitation process

  2022cites this paper
• Facilitating the wise use of experts and evidence to inform local environmental decisions

  2022cites this paper
• Combining Evidence

  2022cites this paper
• Turning inserts the selection approach based on fuzzy comprehensive evaluation

  2022cites this paper
• Towards an empirically informed normative Bayesian scheme-based account of argument from expert opinion

  2022cites this paper
• A systematic method for hypothesis synthesis and conceptual model development

  2022cites this paper
• Attribution of country level foodborne disease to food group and food types in three African countries: Conclusions from a structured expert judgment study

  2022cites this paper
• Quantifying Earth system interactions for sustainable food production via expert elicitation

  2022cites this paper
• The conservation impacts of ecological disturbance: Time‐bound estimates of population loss and recovery for fauna affected by the 2019–2020 Australian megafires

  2022cites this paper
• Consistency in impact assessments of invasive species is generally high and depends on protocols and impact types

  2022cites this paper
• Challenges in estimation, uncertainty quantification and elicitation for pandemic modelling

  2022cites this paper
• Factors affecting expert performance in bid evaluation: An integrated approach

  2022cites this paper
• Can Groups Improve Expert Economic and Financial Forecasts?

  2022cites this paper
• Innovative Expert Methods in Strategic Decision Making

  2022cites this paper
• We Have Sent Ourselves to Iceland (With Apologies to Iceland): Changing the Academy From Internally-Driven to Externally Partnered

  2022cites this paper
• Australia's most imperilled vertebrates

  2022cites this paper
• Using expert elicitation to identify effective combinations of management actions for koala conservation in different regional landscapes

  2022cites this paper
• Vote-processing rules for combining control recommendations from multiple models

  2022cites this paper
• Applying a sequential evidence hierarchy, with caveats, to support prudent fisheries bycatch policy

  2022cites this paper
• Expert perspectives on global biodiversity loss and its drivers and impacts on people

  2022cites this paper
• Improve performance and robustness of knowledge-based FUZZY LOGIC habitat models

  2021cites this paper
• Methods for identifying spatially referenced conservation needs and opportunities

  2021cites this paper
• Balancing making a difference with making a living in the conservation sector

  2021cites this paper
• Using anecdotal reports to clarify the distribution and status of a near mythical species: Australia’s Night Parrot (Pezoporus occidentalis)

  2021cites this paper
• Strengthening conservation science as a crisis discipline by addressing challenges of precaution, privilege, and individualism

  2021cites this paper
• Using Forecasting Methods to Incorporate Social, Economic, and Political Considerations Into Marine Protected Area Planning

  2021cites this paper
• Projecting biodiversity benefits of conservation behavior‐change programs

  2021cites this paper
• Uncertainty Quantification with Experts: Present Status and Research Needs

  2021cites this paper
• Butterflies on the brink: identifying the Australian butterflies (Lepidoptera) most at risk of extinction

  2021cites this paper
• A new framework and tool for ecological risk assessment of wave energy converters projects

  2021cites this paper
• Corrigendum: The legal principles guiding a cohort of early career environmental professionals

  2021cites this paper
• The legal principles guiding a cohort of early career environmental professionals

  2021cites this paper
• Quantifying Earth system interactions for sustainable food production: an expert elicitation

  2021cites this paper
• Collective Intelligence in Organizations

  2021cites this paper
• Predicting species and community responses to global change using structured expert judgement: An Australian mountain ecosystems case study

  2021cites this paper
• Options for reducing uncertainty in impact classification for alien species

  2021cites this paper
• Priority knowledge needs for implementing nature-based solutions in the Mediterranean islands

  2021cites this paper
• Expert Judgement for Geological Hazards in New Zealand

  2021cites this paper
• Use of evidence for decision‐making by conservation practitioners in the illegal wildlife trade

  2021cites this paper