Reviewing versus doing: learning and performance in crowd assessment

No abstract is available for this paper.

• Publication year

  2014

• Venue

  Conference on Computer Supported Cooperative Work

• Publication date

  2014-02-15

• Fields of study

  Computer Science, Psychology

• Identifiers
  DOI 10.1145/2531602.2531718
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Organizing without formal organization: group identification, goal setting and social modeling in directing online production

  2012cited by this paper
• Studying the Effect of Tutor Learning Using a Teachable Agent that Asks the Student Tutor for Explanations

  2012cited by this paper
• Effectiveness of shared leadership in online communities

  2012cited by this paper
• Building Successful Online Communities: Evidence-Based Social Design

  2012cited by this paper
• Shepherding the crowd yields better work

  2012influential reference
• MobileWorks: Designing for Quality in a Managed Crowdsourcing Architecture

  2012cited by this paper
• CrowdForge: crowdsourcing complex work

  2011cited by this paper
• Identifying shared leadership in Wikipedia

  2011cited by this paper
• Employer Expectations, Peer Effects and Productivity: Evidence from a Series of Field Experiments

  2010cited by this paper
• Running experiments on Amazon Mechanical Turk

  2010cited by this paper
• Teachable Agents and the Protégé Effect: Increasing the Effort Towards Learning

  2009cited by this paper
• Crowdsourcing user studies with Mechanical Turk

  2008cited by this paper
• Harnessing the wisdom of crowds in wikipedia: quality through coordination

  2008cited by this paper
• Empowering Leadership in Management Teams: Effects on Knowledge Sharing, Efficacy, And Performance

  2006cited by this paper
• How, When, and Why Bad Apples Spoil the Barrel: Negative Group Members and Dysfunctional Groups

  2006cited by this paper
• Shared leadership: Reframing the hows and whys of leadership

  2006cited by this paper
• Shared Leadership: Reframing the Hows and Whys of Leadership

  2004cited by this paper
• Group performance and decision making.

  2004cited by this paper
• GROUPS : INTERACTION AND PERFORMANCE

  2001cited by this paper
• Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being.

  2000cited by this paper
• The effects of averaging subjective probability estimates between and within judges.

  2000cited by this paper
• Deliberating about Dollars: The Severity Shift

  2000cited by this paper
• Inquiry, Modeling, and Metacognition: Making Science Accessible to All Students

  1998cited by this paper
• Handbook of Child Psychology

  1997cited by this paper
• Brainstorming groups in context: Effectiveness in a product design firm

  1996cited by this paper
• Why Groups are less Effective than their Members: On Productivity Losses in Idea-generating Groups

  1994cited by this paper
• Perception of Performance in Group Brainstorming: The Illusion of Group Productivity

  1993cited by this paper
• Productivity loss in brainstorming groups: A meta-analytic integration.

  1991cited by this paper
• Managing the bossless team: Lessons in distributed leadership

  1991cited by this paper
• Progress in Small Group Research

  1990cited by this paper
• Group polarization: A critical review and meta-analysis.

  1986cited by this paper
• Collective induction: Mutual group and individual influence by exchange of hypotheses and evidence

  1986cited by this paper
• Groupthink: Psychological Studies of Policy Decisions and Fiascoes

  1982cited by this paper
• Learning, remembering, and understanding

  1982cited by this paper
• Leadership in Organizations

  1981cited by this paper
• The Effectiveness of Nominal, Delphi, and Interacting Group Decision Making Processes

  1974cited by this paper
• Nominal Versus Interacting Group Processes for Committee Decision-Making Effectiveness

  1971cited by this paper
• The nature of human intelligence.

  1968cited by this paper
• Evidence for a Collective Intelligence Factor in the Performance of Human Groups

  year unknowncited by this paper

• Hierarchical Worker Evaluation Based on Requester’s Subjective Criteria in Open-Ended Crowdsourcing Tasks

  2026cites this paper
• A Longitudinal Study on Different Annotator Feedback Loops in Complex RAG Tasks

  2025cites this paper
• Feed-O-Meter: Investigating AI-generated mentee personas as interactive agents for scaffolding design feedback practice

  2025cites this paper
• Learning while Earning? A Literature Review and Case Study on Learning Opportunities in Crowdwork

  2024influential citation
• On the Role of Large Language Models in Crowdsourcing Misinformation Assessment

  2024cites this paper
• OSINT Research Studios: A Flexible Crowdsourcing Framework to Scale Up Open Source Intelligence Investigations

  2024cites this paper
• LabelAId: Just-in-time AI Interventions for Improving Human Labeling Quality and Domain Knowledge in Crowdsourcing Systems

  2024cites this paper
• Fostering Data Worker Inclusion and Well-Being: Identifying Barriers and Designing Interventions

  2023cites this paper
• You Know What I Meme: Enhancing People's Understanding and Awareness of Hateful Memes Using Crowdsourced Explanations

  2023cites this paper
• On the Impact of Showing Evidence from Peers in Crowdsourced Truthfulness Assessments

  2023cites this paper
• The Grind for Good Data: Understanding ML Practitioners' Struggles and Aspirations in Making Good Data

  2022cites this paper
• Effects of learning and uncertainty on crowdsourcing performance of solvers: insights from performance feedback theory

  2022cites this paper
• The Influences of Task Design on Crowdsourced Judgement: A Case Study of Recidivism Risk Evaluation

  2022cites this paper
• SmartGroup: A Tool for Small-Group Learning Activities

  2022cites this paper
• Towards AI-Powered Data-Driven Education

  2022cites this paper
• Social media use and work efficiency: Insights from the theory of communication visibility

  2021cites this paper
• Exploring the Effects of Goal Setting When Training for Complex Crowdsourcing Tasks (Extended Abstract)

  2021cites this paper
• A Framework for Open Civic Design: Integrating Public Participation, Crowdsourcing, and Design Thinking

  2021cites this paper
• A Survey on Task Assignment in Crowdsourcing

  2021cites this paper
• Crowd-Worker Skill Improvement with AI Co-Learners

  2021cites this paper
• Task Assignment Strategies for Crowd Worker Ability Improvement

  2021cites this paper
• Computer-Mediated Peer Support Needs of Home Care Workers: Emotional Labor & the Politics of Professionalism

  2021cites this paper
• How We Write with Crowds

  2021cites this paper
• The Challenge of Variable Effort Crowdsourcing and How Visible Gold Can Help

  2021cites this paper
• Optimal Spot-Checking for Improving the Evaluation Quality of Crowdsourcing: Application to Peer Grading Systems

  2020cites this paper
• Shifting forms of Engagement: Volunteer Learning in Online Citizen Science

  2020cites this paper
• Motivating Novice Crowd Workers through Goal Setting: An Investigation into the Effects on Complex Crowdsourcing Task Training

  2020cites this paper
• CrowdMOT

  2020cites this paper
• Supporting Historical Research and Education with Crowdsourced Analysis of Primary Sources

  2019influential citation
• Read-Agree-Predict: A Crowdsourced Approach to Discovering Relevant Primary Sources for Historians

  2019cites this paper
• Distributed Apprenticeship in Online Communities

  2019cites this paper
• Joining Together Online

  2019cites this paper
• Not Everyone Writes Good Examples but Good Examples Can Come from Anywhere

  2019cites this paper
• How Guiding Questions Facilitate Feedback Exchange in Project-Based Learning

  2019cites this paper
• Crowd Coach

  2018cites this paper
• Quality Control in Crowdsourcing

  2018cites this paper
• CrowdLayout: Crowdsourced Design and Evaluation of Biological Network Visualizations

  2018cites this paper
• Distributed Knowledge in Crowds: Crowd Performance on Hidden Profile Tasks

  2018cites this paper
• Information Visualization Evaluation Using Crowdsourcing

  2018cites this paper
• CrowdIA

  2018cites this paper
• Exploring Trade-Offs Between Learning and Productivity in Crowdsourced History

  2018influential citation
• Older Adults and Crowdsourcing

  2018cites this paper
• Working in Pairs: Understanding the Effects of Worker Interactions in Crowdwork

  2018cites this paper
• Exploring product design quality control and assurance under both traditional and crowdsourcing-based design environments

  2018influential citation
• Impact of collaborative team peer review on the quality of feedback in engineering design projects

  2018cites this paper
• Explorer Information Visualization Evaluation Using Crowdsourcing

  2018cites this paper
• A Community Rather Than A Union: Understanding Self-Organization Phenomenon on MTurk and How It Impacts Turkers and Requesters

  2017cites this paper
• On-the-job Learning for Real-time

  2017cites this paper
• Which Size Matters? Effects of Crowd Size on Solution Quality in Big Data Q&A Communities

  2017cites this paper
• Better Organization or a Source of Distraction?: Introducing Digital Peer Feedback to a Paper-Based Classroom

  2017cites this paper
• Supporting Virtual Team Formation through Community-Wide Deliberation

  2017cites this paper
• Efficiently Identifying a Well-Performing Crowd Process for a Given Problem

  2017cites this paper
• SCAFFOLDING CROWD WORK

  2017cites this paper
• Scopist: Building a Skill Ladder into Crowd Transcription

  2017cites this paper
• PeerPresents: A Web-Based System for In-Class Peer Feedback during Student Presentations

  2016cites this paper
• Learning From the Crowd: Observational Learning in Crowdsourcing Communities

  2016cites this paper
• Huddler: Convening Stable and Familiar Crowd Teams Despite Unpredictable Availability

  2016cites this paper
• Personality Matters: Balancing for Personality Types Leads to Better Outcomes for Crowd Teams

  2016cites this paper
• Toward a Learning Science for Complex Crowdsourcing Tasks

  2016influential citation
• Peer review in design: Understanding the impact of collaboration on the review process and student perception

  2016influential citation
• Crowd Guilds: Worker-led Reputation and Feedback on Crowdsourcing Platforms

  2016cites this paper
• Crowdlines: Supporting Synthesis of Diverse Information Sources through Crowdsourced Outlines

  2015cites this paper
• Enhancing Diversity and Coverage of Crowd-Generated Feedback through Social Interaction

  2015cites this paper
• A system for scalable and reliable technical-skill testing in online labor markets

  2015cites this paper
• Structuring Interactions for Large-Scale Synchronous Peer Learning

  2015cites this paper
• Combining Crowdsourcing and Learning to Improve Engagement and Performance

  2014cites this paper
• STEP: A Scalable Testing and Evaluation Platform

  2014cites this paper
• Combining crowdsourcing and learning to improve engagement and performance

  2014cites this paper
• E XPLORING C ROWDSOURCED W ORKER E VALUATION M ETHODSIN O PEN -E NDED T ASKS

  year unknowncites this paper