Human computation tasks with global constraints

An important class of tasks that are underexplored in current human computation systems are complex tasks with global constraints. One example of such a task is itinerary planning, where solutions consist of a sequence of activities that meet requirements specified by the requester. In this paper, we focus on the crowdsourcing of such plans as a case study of constraint-based human computation tasks and introduce a collaborative planning system called Mobi that illustrates a novel crowdware paradigm. Mobi presents a single interface that enables crowd participants to view the current solution context and make appropriate contributions based on current needs. We conduct experiments that explain how Mobi enables a crowd to effectively and collaboratively resolve global constraints, and discuss how the design principles behind Mobi can more generally facilitate a crowd to tackle problems involving global constraints.

• Publication year

  2012

• Venue

  International Conference on Human Factors in Computing Systems

• Publication date

  2012-05-05

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1145/2207676.2207708
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Turkomatic: automatic recursive task and workflow design for mechanical turk

  2011cited by this paper
• Towards Large-Scale Collaborative Planning: Answering High-Level Search Queries Using Human Computation

  2011cited by this paper
• Platemate: crowdsourcing nutritional analysis from food photographs

  2011cited by this paper
• CrowdForge: crowdsourcing complex work

  2011cited by this paper
• Crowdsourcing, collaboration and creativity

  2010cited by this paper
• TurKit: human computation algorithms on mechanical turk

  2010cited by this paper
• Soylent: a word processor with a crowd inside

  2010cited by this paper
• Human computation

  2009cited by this paper
• Mobile opportunistic commerce: mechanisms, architecture, and application

  2008cited by this paper
• Survey of Preference Elicitation Methods

  2004cited by this paper
• Automated Planning: Theory & Practice

  2004cited by this paper
• Groupware: some issues and experiences

  1991cited by this paper
• WYSIWIS revised: early experiences with multiuser interfaces

  1987cited by this paper
• A Cognitive Model of Planning

  1979cited by this paper
• A Structure for Plans and Behavior

  1977cited by this paper

• The Practice of Online Peer Counseling and the Potential for AI-Powered Support Tools

  2025cites this paper
• A Scoping Review of Mixed Initiative Visual Analytics in the Automation Renaissance

  2025cites this paper
• Problem constraints in ideation contests: How different types of constraints matter

  2024cites this paper
• A Human-Computer Collaborative Tool for Training a Single Large Language Model Agent into a Network through Few Examples

  2024cites this paper
• To Err is AI: Imperfect Interventions and Repair in a Conversational Agent Facilitating Group Chat Discussions

  2023cites this paper
• Learning by Teaching: Key Challenges and Design Implications

  2023cites this paper
• Inform, Explain, or Control: Techniques to Adjust End-User Performance Expectations for a Conversational Agent Facilitating Group Chat Discussions

  2023cites this paper
• RetroLens: A Human-AI Collaborative System for Multi-step Retrosynthetic Route Planning

  2023cites this paper
• Towards Collaborative Plan Acquisition through Theory of Mind Modeling in Situated Dialogue

  2023cites this paper
• Characterizing Time Spent in Video Object Tracking Annotation Tasks: A Study of Task Complexity in Vehicle Tracking

  2023cites this paper
• Supporting Global Context under Evolving User Intents during Data Exploration

  2023influential citation
• Designing LLM Chains by Adapting Techniques from Crowdsourcing Workflows

  2023influential citation
• Terratech: A Crowdsourcing Web Application that Aids Environmental Needs and Concerns

  2022cites this paper
• Human-AI Symbiosis: A Survey of Current Approaches

  2021cites this paper
• CrowdFolio

  2021cites this paper
• Supporting Collaborative Sequencing of Small Groups through Visual Awareness

  2021cites this paper
• The Design and Development of a Game to Study Backdoor Poisoning Attacks: The Backdoor Game

  2021cites this paper
• CrowdLens: Experimenting with Crowd-Powered Recommendation and Explanation

  2021cites this paper
• Consensus Building in Collaborative Sequencing with Visual Awareness

  2020cites this paper
• Automated Domain Model Learning Tools for Planning

  2020cites this paper
• CrowdMOT

  2020cites this paper
• Mesh: Scaffolding Comparison Tables for Online Decision Making

  2020cites this paper
• VisiBlends: A Flexible Workflow for Visual Blends

  2019cites this paper
• Considerable Factors for Domain Model Learning in Planning

  2019cites this paper
• Dropping the Baton?

  2019cites this paper
• Macrotask Crowdsourcing: An Integrated Definition

  2019cites this paper
• GroupTravel: Customizing Travel Packages for Groups

  2019cites this paper
• Managing Work Systems for Complex Work via Crowdworking Platforms - How to Orchestrate the Interplay of Crowds

  2019cites this paper
• Critter: Augmenting Creative Work with Dynamic Checklists, Automated Quality Assurance, and Contextual Reviewer Feedback

  2019cites this paper
• Following others through an information maze: The impact of social navigation on information seeking behavior

  2019cites this paper
• Flud: a hybrid crowd-algorithm approach for visualizing biological networks

  2019cites this paper
• Secure and Distributed Crowd-Sourcing Task Coordination Using the Blockchain Mechanism

  2019cites this paper
• Paying Crowd Workers for Collaborative Work

  2019cites this paper
• Understanding crowdsourcing projects: A systematic review of tendencies, workflow, and quality management

  2018cites this paper
• An Interactive Pipeline for Creating Visual Blends

  2018cites this paper
• Online Sequencing of Non-Decomposable Macrotasks in Expert Crowdsourcing

  2018cites this paper
• Creating Better Action Plans for Writing Tasks via Vocabulary-Based Planning

  2018cites this paper
• Hit-or-Wait: Coordinating Opportunistic Low-effort Contributions to Achieve Global Outcomes in On-the-go Crowdsourcing

  2018cites this paper
• Foundations of Human-Aware Planning - A Tale of Three Models

  2018influential citation
• Crowdsourcing Intelligence Analysis with Context Slices

  2018cites this paper
• Spam elimination and bias correction : ensuring label quality in crowdsourced tasks.

  2018influential citation
• Crowdsourcing Exercise Plans Aligned with Expert Guidelines and Everyday Constraints

  2018cites this paper
• Harvesting Caregiving Knowledge: Design Considerations for Integrating Volunteer Input in Dementia Care

  2018cites this paper
• CrowdIA

  2018cites this paper
• CommunityCrit: Inviting the Public to Improve and Evaluate Urban Design Ideas through Micro-Activities

  2018cites this paper
• Human based Computation: A solution to Computer Vision Problems

  2018cites this paper
• Towards Intelligent Decision Support in Human Team Planning

  2017cites this paper
• Herding the Crowd: Using Automated Planning for Better Crowdsourced Planning

  2017influential citation
• Interactive system with explanation function for real‐time railway crew rescheduling

  2017cites this paper
• INForum 2017: Atas do Nono Simpósio de Informática

  2017cites this paper
• CYTRIP: A Multi-day Trip Planning System based on Crowdsourced POIs Recommendation

  2017cites this paper
• Putting the Pieces Back Together Again: Contest Webs for Large-Scale Problem Solving

  2017cites this paper
• Argument discovery via crowdsourcing

  2017cites this paper
• Calendar.help: Designing a Workflow-Based Scheduling Agent with Humans in the Loop

  2017cites this paper
• Customizing Travel Packages with Interactive Composite Items

  2017cites this paper
• Kurator: Using The Crowd to Help Families With Personal Curation Tasks

  2017cites this paper
• Computational design of mixed-initiative human–robot teaming that considers human factors: situational awareness, workload, and workflow preferences

  2017influential citation
• Communicating Context to the Crowd for Complex Writing Tasks

  2017cites this paper
• Flash Organizations: Crowdsourcing Complex Work by Structuring Crowds As Organizations

  2017cites this paper
• Complementing the Execution of AI Systems with Human Computation

  2017cites this paper
• Making Better Use of the Crowd: How Crowdsourcing Can Advance Machine Learning Research

  2017cites this paper
• Towards Scalable Governance: Sensemaking and Cooperation in the Age of Social Media

  2016cites this paper
• Using the Crowd to Improve Search Result Ranking and the Search Experience

  2016cites this paper
• Crowd-sourcing Literal Review

  2016cites this paper
• PlanSourcing: Generating Behavior Change Plans with Friends and Crowds

  2016cites this paper
• Leveraging Mixed Expertise in Crowdsourcing

  2016cites this paper
• Toward a Quantitative Understanding of Teamwork and Collective Intelligence

  2016cites this paper
• IdeaHound: Improving Large-scale Collaborative Ideation with Crowd-Powered Real-time Semantic Modeling

  2016cites this paper
• Large-Scale Collaborative Innovation: Challenges, Visions and Approaches

  2016cites this paper
• From Books to Bots: Using Medical Literature to Create a Chat Bot

  2016cites this paper
• Productivity Decomposed: Getting Big Things Done with Little Microtasks

  2016cites this paper
• Supporting Collaborative Writing with Microtasks

  2016cites this paper
• Human computation for data, information, and knowledge management

  2016cites this paper
• Relational Crowdsourcing and its Application in Knowledge Graph Evaluation

  2016cites this paper
• MicroTalk: Using Argumentation to Improve Crowdsourcing Accuracy

  2016cites this paper
• Symptom-Diagnosis-Care: A Framework for a Collaborative Medical Chat Bot

  2016cites this paper
• Multi-day Trip Planning System with Collaborative Recommendation*

  2016cites this paper
• The Knowledge Accelerator: Big Picture Thinking in Small Pieces

  2016cites this paper
• WEATHER LORE VALIDATION TOOL USING FUZZY COGNITIVE MAPS BASED ON COMPUTER VISION

  2016cites this paper
• Mechanical Novel: Crowdsourcing Complex Work through Reflection and Revision

  2016cites this paper
• Avicaching: A Two Stage Game for Bias Reduction in Citizen Science

  2016cites this paper
• KGEval: Estimating Accuracy of Automatically Constructed Knowledge Graphs

  2016cites this paper
• Collaborative Solving in a Human Computing Game Using a Market, Skills and Challenges

  2016cites this paper
• Toward a Learning Science for Complex Crowdsourcing Tasks

  2016cites this paper
• Crowdlines: Supporting Synthesis of Diverse Information Sources through Crowdsourced Outlines

  2015cites this paper
• Crowdsourcing Techniques for Affective Computing

  2015cites this paper
• Supporting Collaborative Innovation at Scale

  2015cites this paper
• Matching and Grokking: Approaches to Personalized Crowdsourcing

  2015cites this paper
• Designing for Collaborative Sensemaking: Using Expert & Non-Expert Crowd

  2015cites this paper
• Argonaut: Macrotask Crowdsourcing for Complex Data Processing

  2015cites this paper
• Crowd Agents: Interactive Intelligent Systems Powered by the Crowd

  2015cites this paper
• PhD Thesis Proposal: Human-Machine Collaborative Optimization via Apprenticeship Scheduling

  2015cites this paper
• Intelligent Systems to Support Large-Scale Collective Creative Idea Generation

  2015cites this paper
• Planning with Humans in the Loop Tathagata Chakraborti

  2015cites this paper
• HumorTools : A Microtask Workflow for Writing News Satire

  2015cites this paper
• Temporal modeling of crowd work quality for quality assurance in crowdsourcing

  2015cites this paper
• Measuring text simplification with the crowd

  2015cites this paper
• Crowdsourced Action-Model Acquisition for Planning

  2015cites this paper
• An Experiment in Microtask Crowdsourcing Software Design

  2015cites this paper
• Crowdsourced Data Management: Industry and Academic Perspectives

  2015cites this paper