The Yale human grasping dataset: Grasp, object, and task data in household and machine shop environments

This paper presents a dataset of human grasping behavior in unstructured environments. Wide-angle head-mounted camera video was recorded from two housekeepers and two machinists during their regular work activities, and the grasp types, objects, and tasks were analyzed and coded by study staff. The full dataset contains 27.7 hours of tagged video and represents a wide range of manipulative behaviors spanning much of the typical human hand usage. We provide the original videos, a spreadsheet including the tagged grasp type, object, and task parameters, time information for each successive grasp, and video screenshots for each instance. Example code is provided for MATLAB and R, demonstrating how to load in the dataset and produce simple plots.

• Publication year

  2015

• Venue

  Int. J. Robotics Res.

• Publication date

  2015-03-01

• Fields of study

  Computer Science, Engineering

• Identifiers
  DOI 10.1177/0278364914555720
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Creative Commons Attribution 4.0 International (CC BY 4.0)

  2015cited by this paper
• Analysis of Human Grasping Behavior: Correlating Tasks, Objects and Grasps

  2014influential reference
• Analysis of Human Grasping Behavior: Object Characteristics and Grasp Type

  2014influential reference
• Grasp Frequency and Usage in Daily Household and Machine Shop Tasks

  2013influential reference
• A comprehensive grasp taxonomy

  2009cited by this paper
• A wearable system that learns a kinematic model and finds structure in everyday manipulation by using absolute orientation sensors and a camera

  2005cited by this paper
• On grasp choice, grasp models, and the design of hands for manufacturing tasks

  1989cited by this paper
• Kappa Reliabilities for Continuous Behaviors and Events

  1985cited by this paper
• A Coefficient of Agreement for Nominal Scales

  1960cited by this paper
• Beitrag zur Schotteranalyse

  1935cited by this paper

• NVS-HO: A Benchmark for Novel View Synthesis of Handheld Objects

  2026cites this paper
• Enhancing robotic skill acquisition with multimodal sensory data: A novel dataset for kitchen tasks

  2025cites this paper
• Task-Oriented Grasping Using Reinforcement Learning with a Contextual Reward Machine

  2025cites this paper
• Robustness of PCANet with Block Optimization in Noisy Image Recognition

  2025cites this paper
• Multi-strategy improvement of crayfish optimization algorithm to solve high-dimensional feature selection

  2025cites this paper
• Mid-Air Gestures for Proactive Olfactory Interactions in Virtual Reality

  2025cites this paper
• Advancing robots with greater dynamic dexterity: A large-scale multi-view and multi-modal dataset of human-human throw&catch of arbitrary objects

  2024cites this paper
• HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos

  2024cites this paper
• BiCap: A novel bi-modal dataset of daily living dual-arm manipulation actions

  2024cites this paper
• An Exploratory Study on the Relation between Grasp Types and Bimanual Categories in Manipulation Activities

  2024cites this paper
• SHOWMe: Robust object-agnostic hand-object 3D reconstruction from RGB video

  2024cites this paper
• Introducing HOT3D: An Egocentric Dataset for 3D Hand and Object Tracking

  2024cites this paper
• VR-Grasp: A Human Grasp Taxonomy for Virtual Reality

  2024cites this paper
• HANDdata – first-person dataset including proximity and kinematics measurements from reach-to-grasp actions

  2023cites this paper
• Get a Grip: Reconstructing Hand-Object Stable Grasps in Egocentric Videos

  2023cites this paper
• Design and implementation of an underactuated gripper with enhanced shape adaptability and lateral stiffness through semi-active multi-degree-of-freedom endoskeletons

  2023cites this paper
• BaRiFlex: A Robotic Gripper with Versatility and Collision Robustness for Robot Learning

  2023cites this paper
• MANUS: Markerless Grasp Capture Using Articulated 3D Gaussians

  2023cites this paper
• Two-Step Self-Supervised Pretraining Approach for Machinist Hand Gesture Recognition

  2023cites this paper
• A survey of deep learning methods and datasets for hand pose estimation from hand-object interaction images

  2023cites this paper
• SHOWMe: Benchmarking Object-agnostic Hand-Object 3D Reconstruction

  2023cites this paper
• How the hand has shaped sign languages

  2022cites this paper
• A kinematic and EMG dataset of online adjustment of reach-to-grasp movements to visual perturbations

  2022cites this paper
• Object-Centric Grasping Transferability: Linking Meshes to Postures

  2022cites this paper
• Multi-Finger Haptics: Analysis of Human Hand Grasp towards a Tripod Three-Finger Haptic Grasp model

  2022cites this paper
• Robotic Grasping from Classical to Modern: A Survey

  2022cites this paper
• Relationship between Grasping Actions and Object Attributes: A Survey

  2021cites this paper
• Text-driven object affordance for guiding grasp-type recognition in multimodal robot teaching

  2021cites this paper
• Quantitative Investigation of Hand Grasp Functionality: Hand Joint Motion Correlation, Independence, and Grasping Behavior

  2021cites this paper
• Ego-Interaction: Visual Hand-Object Pose Correction for VR Experiences

  2021cites this paper
• Pose measurement and assembly of spacecraft components based on assembly features and a consistent coordinate system

  2021cites this paper
• Learning Task-Oriented Dexterous Grasping from Human Knowledge

  2021cites this paper
• Ego4D: Around the World in 3,000 Hours of Egocentric Video

  2021cites this paper
• A comprehensive grasp taxonomy of continuum robots

  2021cites this paper
• Foreground-Aware Stylization and Consensus Pseudo-Labeling for Domain Adaptation of First-Person Hand Segmentation

  2021cites this paper
• The KIT Bimanual Manipulation Dataset

  2021cites this paper
• Object affordance as a guide for grasp-type recognition

  2021cites this paper
• Too Hot to Handle: An Evaluation of the Effect of Thermal Visual Representation on User Grasping Interaction in Virtual Reality

  2020cites this paper
• A First-person Database for Detecting Barriers for Pedestrians

  2020cites this paper
• Event-based Robotic Grasping Detection with Neuromorphic Vision Sensor and Event-Stream Dataset

  2020cites this paper
• Generalizing Hand Segmentation in Egocentric Videos With Uncertainty-Guided Model Adaptation

  2020cites this paper
• GanHand: Predicting Human Grasp Affordances in Multi-Object Scenes

  2020cites this paper
• Analysis of subject specific grasping patterns

  2020cites this paper
• Technology for monitoring everyday prosthesis use: a systematic review

  2020cites this paper
• ContactPose: A Dataset of Grasps with Object Contact and Hand Pose

  2020cites this paper
• Grasp-type Recognition Leveraging Object Affordance

  2020cites this paper
• Event-Based Robotic Grasping Detection With Neuromorphic Vision Sensor and Event-Grasping Dataset

  2020cites this paper
• Ego2Hands: A Dataset for Egocentric Two-hand Segmentation and Detection

  2020cites this paper
• Knowledge-Augmented Dexterous Grasping with Incomplete Sensing

  2020cites this paper
• Human-machine shared control: New avenue to dexterous prosthetic hand manipulation

  2020cites this paper
• The MoCA dataset, kinematic and multi-view visual streams of fine-grained cooking actions

  2020cites this paper
• Video Grasping Classification Enhanced with Automatic Annotations

  2020cites this paper
• Detection and Recognition of Barriers in Egocentric Images for Safe Urban Sidewalks

  2020cites this paper
• Attention Based Visual Analysis for Fast Grasp Planning With a Multi-Fingered Robotic Hand

  2019cites this paper
• Human hand kinematic data during feeding and cooking tasks

  2019cites this paper
• WorkingHands: A Hand-Tool Assembly Dataset for Image Segmentation and Activity Mining

  2019cites this paper
• HO-3D: A Multi-User, Multi-Object Dataset for Joint 3D Hand-Object Pose Estimation

  2019cites this paper
• Grasping Microgestures: Eliciting Single-hand Microgestures for Handheld Objects

  2019cites this paper
• Data-Driven Video Grasping Classification for Low-Power Embedded System

  2019cites this paper
• A benchmark image dataset for industrial tools

  2019cites this paper
• Mask-Pose Cascaded CNN for 2D Hand Pose Estimation From Single Color Image

  2019cites this paper
• ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging

  2019cites this paper
• A Survey of Methods and Strategies for High-Precision Robotic Grasping and Assembly Tasks—Some New Trends

  2019cites this paper
• Integrating Perception and Optimization for Dexterous Grasping and Manipulation

  2019cites this paper
• HOnnotate: A Method for 3D Annotation of Hand and Object Poses

  2019cites this paper
• Hand design priority based on cadence of grasping direction in deskwork

  2019cites this paper
• Classifying Object Manipulation Actions based on Grasp-types and Motion-Constraints

  2018influential citation
• Human grasping database for activities of daily living with depth, color and kinematic data streams

  2018cites this paper
• Partially Occluded Hands: - A Challenging New Dataset for Single-Image Hand Pose Estimation

  2018cites this paper
• Improving the functionality, robustness, and adaptability of myoelectric control for dexterous motion restoration

  2018cites this paper
• Long Term Hand Tracking with Proposal Selection

  2018cites this paper
• Attention based visual analysis for fast grasp planning with multi-fingered robotic hand

  2018cites this paper
• Multimodal Tactile Perception of Objects in a Real Home

  2018cites this paper
• Linking Elementary Properties of the Human Brain to the Behaviour Captured on Touchscreen Smartphones

  2017cites this paper
• First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations

  2017influential citation
• Hand pose estimation and motion recognition using egocentric RGB-D video

  2017cites this paper
• A new dataset for hand gesture estimation

  2017cites this paper
• Assistive mobile manipulation for users with severe motor impairments

  2017cites this paper
• Biomimetic Based EEG Learning for Robotics Complex Grasping and Dexterous Manipulation

  2017cites this paper
• An Ego-Vision System for Hand Grasp Analysis

  2017cites this paper
• The complexities of grasping in the wild

  2017cites this paper
• Long-Term Tracking by Decision Making

  2017influential citation
• A novel actuation configuration of robotic hand and the mechanical implementation via postural synergies

  2017cites this paper
• Grasp type understanding — classification, localization and clustering

  2016cites this paper
• Comparison between grasp quality metrics and the anthropomorphism index for the evaluation of artificial hands

  2016cites this paper
• Recent Data Sets on Object Manipulation: A Survey

  2016influential citation
• Prediction of Manipulation Actions

  2016cites this paper
• The GRASP Taxonomy of Human Grasp Types

  2016influential citation
• Analysis on the joint independence of hand and wrist

  2016cites this paper
• CapriDB - Capture, Print, Innovate: A Low-Cost Pipeline and Database for Reproducible Manipulation Research

  2016cites this paper
• Towards Hand-Object Gesture Extraction from Depth Image

  2016cites this paper
• Movement Primitive Segmentation for Human Motion Modeling: A Framework for Analysis

  2016cites this paper
• Effectiveness of Grasp Attributes and Motion-Constraints for Fine-Grained Recognition of Object Manipulation Actions

  2016influential citation
• An actuation configuration of inter-module coordination and the evaluation for the mechanical implementation to a prosthetic hand

  2016cites this paper
• 3DCOMET: 3D compression methods test dataset

  2016cites this paper
• Datasets on object manipulation and interaction: a survey

  2016influential citation
• Modeling Grasp Motor Imagery

  2016cites this paper
• Development of an artificial muscle for a soft robotic hand prosthesis

  2016cites this paper
• How do we use our hands? Discovering a diverse set of common grasps

  2015cites this paper
• A scalable approach for understanding the visual structures of hand grasps

  2015cites this paper