Egocentric activity recognition using Histograms of Oriented Pairwise Relations

This paper presents an approach for recognising activities using video from an egocentric (first-person view) setup. Our approach infers activity from the interactions of objects and hands. In contrast to previous approaches to activity recognition, we do not require to use an intermediate such as object detection, pose estimation, etc. Recently, it has been shown that modelling the spatial distribution of visual words corresponding to local features further improves the performance of activity recognition using the bag-of-visual words representation. Influenced and inspired by this philosophy, our method is based on global spatio-temporal relationships between visual words. We consider the interaction between visual words by encoding their spatial distances, orientations and alignments. These interactions are encoded using a histogram that we name the Histogram of Oriented Pairwise Relations (HOPR). The proposed approach is robust to occlusion and background variation and is evaluated on two challenging egocentric activity datasets consisting of manipulative task. We introduce a novel representation of activities based on interactions of local features and experimentally demonstrate its superior performance in comparison to standard activity representations such as bag-of-visual words.

• Publication year

  2014

• Venue

  International Conference on Computer Vision Theory and Applications

• Publication date

  2014-01-05

• Fields of study

  Computer Science

• Identifiers
  DOI 10.5220/0004655100220030
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Workflow Activity Monitoring Using Dynamics of Pair-Wise Qualitative Spatial Relations

  2012influential reference
• Egocentric Activity Monitoring and Recovery

  2012influential reference
• Efficient Additive Kernels via Explicit Feature Maps

  2012cited by this paper
• Fast unsupervised ego-action learning for first-person sports videos

  2011cited by this paper
• HMDB: A large video database for human motion recognition

  2011cited by this paper
• Understanding egocentric activities

  2011influential reference
• Novelty detection from an ego-centric perspective

  2011cited by this paper
• Learning to recognize objects in egocentric activities

  2011influential reference
• Representing Pairwise Spatial and Temporal Relations for Action Recognition

  2010cited by this paper
• Global and efficient self-similarity for object classification and detection

  2010cited by this paper
• Notes on the OpenSURF Library

  2009influential reference
• Spatio-temporal relationship match: Video structure comparison for recognition of complex human activities

  2009influential reference
• Fast realistic multi-action recognition using mined dense spatio-temporal features

  2009cited by this paper
• Hierarchical spatio-temporal context modeling for action recognition

  2009cited by this paper
• Covariant Policy Search

  2009cited by this paper
• Recognizing realistic actions from videos “in the wild”

  2009cited by this paper
• LIBLINEAR: A Library for Large Linear Classification

  2008cited by this paper
• Integrated feature selection and higher-order spatial feature extraction for object categorization

  2008cited by this paper
• Machine Recognition of Human Activities: A Survey

  2008cited by this paper
• Learning realistic human actions from movies

  2008cited by this paper
• A Hierarchical Model of Shape and Appearance for Human Action Classification

  2007cited by this paper
• Matching Local Self-Similarities across Images and Videos

  2007cited by this paper
• Objects in Action: An Approach for Combining Action Understanding and Object Perception

  2007cited by this paper
• Sparse Flexible Models of Local Features

  2006cited by this paper
• Weakly Supervised Learning of Part-Based Spatial Models for Visual Object Recognition

  2006cited by this paper
• Discriminative Object Class Models of Appearance and Shape by Correlatons

  2006cited by this paper
• Actions as space-time shapes

  2005influential reference
• On Space-Time Interest Points

  2005influential reference
• Behavior recognition via sparse spatio-temporal features

  2005cited by this paper
• Recognizing human actions: a local SVM approach

  2004influential reference
• Distinctive Image Features from Scale-Invariant Keypoints

  2004cited by this paper
• Distinctive Image Features from Scale-Invariant Keypoints

  2004influential reference
• A New Tractable Subclass of the Rectangle Algebra

  1999cited by this paper
• Real-time American Sign Language recognition from video using hidden Markov models

  1995cited by this paper
• Maintaining knowledge about temporal intervals

  1983cited by this paper
• Non-commercial Research and Educational Use including without Limitation Use in Instruction at Your Institution, Sending It to Specific Colleagues That You Know, and Providing a Copy to Your Institution's Administrator. All Other Uses, Reproduction and Distribution, including without Limitation Comm

  year unknowncited by this paper

• High-Order Evolving Graphs for Enhanced Representation of Traffic Dynamics

  2024cites this paper
• Driving Through Graphs: a Bipartite Graph for Traffic Scene Analysis

  2024cites this paper
• Exploiting Egocentric Cues for Action Recognition for Ambient Assisted Living Applications

  2021cites this paper
• Egocentric Vision-based Action Recognition: A survey

  2021cites this paper
• Exploiting Three-Dimensional Gaze Tracking for Action Recognition During Bimanual Manipulation to Enhance Human–Robot Collaboration

  2018cites this paper
• Recognition of Activities of Daily Living with Egocentric Vision: A Review

  2016cites this paper
• Egocentric Activity Recognition Using Bag of Visual Words

  2016cites this paper
• Video content analysis on body-worn cameras for retrospective investigation

  2015cites this paper