Evaluation in the Crowd. Crowdsourcing and Human-Centered Experiments

No abstract is available for this paper.

• Publication year

  2017

• Venue

  Lecture Notes in Computer Science

• Publication date

  Unknown publication date

• Fields of study

  Computer Science, Psychology

• Identifiers
  DOI 10.1007/978-3-319-66435-4
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Understanding Comics: The Invisible Art

  2018cited by this paper
• Quality Management in Crowdsourcing using Gold Judges Behavior

  2016cited by this paper
• VEEVVIE: Visual Explorer for Empirical Visualization, VR and Interaction Experiments

  2016cited by this paper
• HOLA: Human-like Orthogonal Network Layout

  2016influential reference
• Comparing Node‐Link and Node‐Link‐Group Visualizations From An Enjoyment Perspective

  2016cited by this paper
• QoE in the Web: A dance of design and performance

  2015cited by this paper
• Detecting low-quality crowdtesting workers

  2015cited by this paper
• Bridging the gap between eye tracking and crowdsourcing

  2015cited by this paper
• Towards Understanding Enjoyment and Flow in Information Visualization

  2015cited by this paper
• AWARE: Mobile Context Instrumentation Framework

  2015influential reference
• GraphUnit: Evaluating Interactive Graph Visualizations Using Crowdsourcing

  2015cited by this paper
• Mobile crowdsourcing [Editor's note]

  2015cited by this paper
• How Do Display Design and User Characteristics Matter in Animations?: An Empirical Study with Air Traffic Control Displays

  2015cited by this paper
• Crowdsourcing facial responses to online videos: Extended abstract

  2015cited by this paper
• An Exploratory Study of Data Sketching for Visual Representation

  2015cited by this paper
• Stock Lamp: An Engagement-Versatile Visualization Design

  2015cited by this paper
• Factors influencing the decision to crowdsource: A systematic literature review

  2015influential reference
• TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking

  2015cited by this paper
• We Are Dynamo: Overcoming Stalling and Friction in Collective Action for Crowd Workers

  2015influential reference
• Towards a Taxonomy for Evaluating User Engagement in Information Visualization

  2015cited by this paper
• How Deceptive are Deceptive Visualizations?: An Empirical Analysis of Common Distortion Techniques

  2015cited by this paper
• Eye Tracker in the Wild: Studying the delta between what is said and measured in a crowdsourcing experiment

  2015cited by this paper
• Understand users’ comprehension and preferences for composing information visualizations

  2014cited by this paper
• Nonparametric Generation of Synthetic Data for Small Geographic Areas

  2014cited by this paper
• Generating synthetic data to produce public-use microdata for small geographic areas based on complex sample survey data with application to the National Health Interview Survey

  2014cited by this paper
• Best Practices for QoE Crowdtesting: QoE Assessment With Crowdsourcing

  2014influential reference
• Privacy for Peer Profiling in Collective Adaptive Systems

  2014influential reference
• Being a turker

  2014influential reference
• Triangulating Eye Movement Data of Animated Displays

  2014cited by this paper
• Embodied decision making with animations

  2014cited by this paper
• Predicting result quality in Crowdsourcing using application layer monitoring

  2014influential reference
• Mimic: visual analytics of online micro-interactions

  2014cited by this paper
• A semantic obfuscation technique for the Internet of Things

  2014influential reference
• Is “chart junk” useful? An extended examination of visual embellishment

  2014cited by this paper
• Crowdsourcing a HIT: Measuring Workers' Pre-Task Interactions on Microtask Markets

  2013cited by this paper
• The future of video analytics for surveillance and its ethical implications

  2013cited by this paper
• An Exploratory Factor Analysis of Motivations for Participating in Zooniverse, a Collection of Virtual Citizen Science Projects

  2013influential reference
• Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral Research

  2013influential reference
• Towards an understanding of visual appeal in website design

  2013cited by this paper
• Using fNIRS brain sensing to evaluate information visualization interfaces

  2013cited by this paper
• Beyond AMT: An Analysis of Crowd Work Platforms

  2013cited by this paper
• jActivity: supporting mobile web developers with HTML5/JavaScript based human activity recognition

  2013cited by this paper
• How users interact with a 3D geo-browser under time pressure

  2013cited by this paper
• Exploring augmented reality for user-generated hyperlocal news content

  2013cited by this paper
• Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers

  2013cited by this paper
• Lab experiment vs. crowdsourcing: a comparative user study on Skype call quality

  2013cited by this paper
• Volunteering Versus Work for Pay: Incentives and Tradeoffs in Crowdsourcing

  2013influential reference
• Turkopticon: interrupting worker invisibility in amazon mechanical turk

  2013influential reference
• The future of crowd work

  2013influential reference
• Data collection in a flat world: the strengths and weaknesses of mechanical turk samples

  2013cited by this paper
• Increasing Payments in Crowdsourcing: Don’t look a gift horse in the mouth!

  2013cited by this paper
• Crowdsourcing Multimedia QoE Evaluation: A Trusted Framework

  2013cited by this paper
• SketchPadN-D: WYDIWYG Sculpting and Editing in High-Dimensional Space

  2013cited by this paper
• An empirical study on the impact of edge bundling on user comprehension of graphs

  2012cited by this paper
• Mechanical Cheat: Spamming Schemes and Adversarial Techniques on Crowdsourcing Platforms

  2012cited by this paper
• Experimental Human-Computer Interaction: Data collection and qualitative analysis

  2012cited by this paper
• Mouse tracking: measuring and predicting users' experience of web-based content

  2012cited by this paper
• Assessing the Effect of Visualizations on Bayesian Reasoning through Crowdsourcing

  2012cited by this paper
• Evaluating the Effect of Style in Information Visualization

  2012cited by this paper
• Empirical Studies in Information Visualization: Seven Scenarios

  2012cited by this paper
• Science & Technology

  2012cited by this paper
• Eulerian video magnification for revealing subtle changes in the world

  2012cited by this paper
• HodgeRank on Random Graphs for Subjective Video Quality Assessment

  2012cited by this paper
• Triumphs of Experience

  2012cited by this paper
• Memorability of Visual Features in Network Diagrams

  2012cited by this paper
• Quality through flow and immersion: gamifying crowdsourced relevance assessments

  2012cited by this paper
• How to Analyze Paired Comparison Data

  2011cited by this paper
• Working the Crowd: Employment and Labor Law in the Crowdsourcing Industry

  2011influential reference
• Maverick: Providing Web Applications with Safe and Flexible Access to Local Devices

  2011cited by this paper
• Crowdsourced data collection of facial responses

  2011influential reference
• Anatomy of a Crowdsourcing Platform - Using the Example of Microworkers.com

  2011cited by this paper
• Evaluating speech synthesis intelligibility using Amazon Mechanical Turk

  2010cited by this paper
• Financial incentives and the "performance of crowds"

  2010cited by this paper
• Author Manuscript, Published in "international Conference on Human Factors in Computing Systems Touchstone: Exploratory Design of Experiments

  2010cited by this paper
• Exploring iterative and parallel human computation processes

  2010cited by this paper
• Synthetic Data for Small Area Estimation

  2010cited by this paper
• Who are the crowdworkers?: shifting demographics in mechanical turk

  2010cited by this paper
• Online labour markets: an inquiry into oDesk providers

  2010cited by this paper
• Graph Generation with Prescribed Feature Constraints

  2009cited by this paper
• Movement data anonymity through generalization

  2009cited by this paper
• A Nested Model for Visualization Design and Validation

  2009cited by this paper
• Galaxy Zoo: Exploring the Motivations of Citizen Science Volunteers.

  2009cited by this paper
• The Evolution of Video Quality Measurement: From PSNR to Hybrid Metrics

  2008cited by this paper
• Visualizing graphs in three dimensions

  2008cited by this paper
• Robust De-anonymization of Large Sparse Datasets

  2008influential reference
• A taxonomy of tasks for guiding the evaluation of multidimensional visualizations

  2006cited by this paper
• Reliability and validity of a computerized neurocognitive test battery, CNS Vital Signs.

  2006cited by this paper
• Task taxonomy for graph visualization

  2006cited by this paper
• The predictive validity of cognitive ability tests: A UK meta-analysis

  2005influential reference
• A Matlab function to estimate choice model parameters from paired-comparison data

  2004cited by this paper
• The end of the rainbow? Color schemes for improved data graphics

  2004cited by this paper
• Motivating devoted workers

  2004cited by this paper
• The challenge of information visualization evaluation

  2004cited by this paper
• Cognitive Load Measurement as a Means to Advance Cognitive Load Theory

  2003cited by this paper
• On the Generation of Spatiotemporal Datasets

  1999cited by this paper
• Which Aesthetic has the Greatest Effect on Human Understanding?

  1997cited by this paper
• The eyes have it: a task by data type taxonomy for information visualizations

  1996cited by this paper
• What Do Bureaucrats Do? The Effects of Performance Standards and Bureaucratic Preferences on Acceptance into the Jtpa Program

  1996influential reference
• THE BIG FIVE PERSONALITY DIMENSIONS AND JOB PERFORMANCE: A META-ANALYSIS

  1991cited by this paper
• A Test of Graphicacy in Children

  1980cited by this paper

• Reviewing Crowdsourcing and Community Engagement in Museums

  2026cites this paper
• Seeing Through the Overlap: The Impact of Color and Opacity on Depth Order Perception in Visualization

  2025cites this paper
• Impact of incidental visualizations on primary tasks

  2023cites this paper
• Subjective Assessment of Objective Image Quality Metrics Range Guaranteeing Visually Lossless Compression

  2023cites this paper
• Quality assessment of higher resolution images and videos with remote testing

  2023cites this paper
• The Dark Side of Recruitment in Crowdsourcing: Ethics and Transparency in Micro-Task Marketplaces

  2023cites this paper
• An Interdisciplinary Perspective on Evaluation and Experimental Design for Visual Text Analytics: Position Paper

  2022cites this paper
• Coding of Still Pictures

  2022cites this paper
• Understanding crowdsourcing in science

  2022cites this paper
• Crowdsourcing Creative Work

  2022cites this paper
• Systematic Analysis of Experiment Precision Measures and Methods for Experiments Comparison

  2022cites this paper
• Towards fair and pro-social employment of digital pieceworkers for sourcing machine learning training data

  2022cites this paper
• Comparative Evaluation of Bipartite, Node-Link, and Matrix-Based Network Representations

  2022cites this paper
• Bringing Graph Databases and Network Visualization Together (Dagstuhl Seminar 22031)

  2022cites this paper
• AVrate Voyager: an open source online testing platform

  2021cites this paper
• Towards Visual Sociolinguistic Network Analysis

  2021cites this paper
• Bigger Isn’t Better: The Ethical and Scientific Vices of Extra-Large Datasets in Language Models

  2021cites this paper
• An Efficient Decision Support System for the Selection of Appropriate Crowd in Crowdsourcing

  2021cites this paper
• Predicting intent behind selections in scatterplot visualizations

  2021cites this paper
• Reliability of Crowdsourcing for Subjective Quality Evaluation of Tone Mapping Operators

  2021cites this paper
• Towards speech quality assessment using a crowdsourcing approach: evaluation of standardized methods

  2020cites this paper
• CrowdWatcher: an open-source platform to catch the eye of the crowd

  2019cites this paper
• Background Environment Characteristics of Crowd-Workers from German Speaking Countries Experimental Survey on User Environment Characteristics

  2019influential citation
• Node-Link or Adjacency Matrices: Old Question, New Insights

  2019cites this paper
• Information Visualization Evaluation Using Crowdsourcing

  2018cites this paper
• Explorer Information Visualization Evaluation Using Crowdsourcing

  2018cites this paper
• (re)Considering Bertin in the age of big data and visual analytics

  2018cites this paper
• Personalized Motivation-supportive Messages for Increasing Participation in Crowd-civic Systems

  2018cites this paper
• Revisited Experimental Comparison of Node-Link and Matrix Representations

  2017cites this paper