The First Facial Landmark Tracking in-the-Wild Challenge: Benchmark and Results

Detection and tracking of faces in image sequences is among the most well studied problems in the intersection of statistical machine learning and computer vision. Often, tracking and detection methodologies use a rigid representation to describe the facial region 1, hence they can neither capture nor exploit the non-rigid facial deformations, which are crucial for countless of applications (e.g., facial expression analysis, facial motion capture, high-performance face recognition etc.). Usually, the non-rigid deformations are captured by locating and tracking the position of a set of fiducial facial landmarks (e.g., eyes, nose, mouth etc.). Recently, we witnessed a burst of research in automatic facial landmark localisation in static imagery. This is partly attributed to the availability of large amount of annotated data, many of which have been provided by the first facial landmark localisation challenge (also known as 300-W challenge). Even though now well established benchmarks exist for facial landmark localisation in static imagery, to the best of our knowledge, there is no established benchmark for assessing the performance of facial landmark tracking methodologies, containing an adequate number of annotated face videos. In conjunction with ICCV'2015 we run the first competition/challenge on facial landmark tracking in long-term videos. In this paper, we present the first benchmark for long-term facial landmark tracking, containing currently over 110 annotated videos, and we summarise the results of the competition.

• Publication year

  2015

• Venue

  2015 IEEE International Conference on Computer Vision Workshop (ICCVW)

• Publication date

  2015-12-01

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1109/ICCVW.2015.132
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• 300 Faces In-The-Wild Challenge: database and results

  2016influential reference
• NUS-PRO: A New Visual Tracking Challenge

  2016cited by this paper
• A survey on face detection in the wild: Past, present and future

  2015cited by this paper
• Transportation Research Board of the National Academies

  2015cited by this paper
• Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition

  2015cited by this paper
• Offline Deformable Face Tracking in Arbitrary Videos

  2015cited by this paper
• Object Tracking Benchmark

  2015cited by this paper
• Unifying holistic and Parts-Based Deformable Model fitting

  2015cited by this paper
• Face Flow

  2015cited by this paper
• Active Pictorial Structures

  2015cited by this paper
• Multi-view Constrained Local Models for Large Head Angle Facial Tracking

  2015cited by this paper
• Shape Augmented Regression Method for Face Alignment

  2015cited by this paper
• Feature-Based Lucas–Kanade and Active Appearance Models

  2015cited by this paper
• Global supervised descent method

  2015influential reference
• Project-Out Cascaded Regression with an application to face alignment

  2015cited by this paper
• From Pixels to Response Maps: Discriminative Image Filtering for Face Alignment in the Wild

  2015cited by this paper
• Facial Shape Tracking via Spatio-Temporal Cascade Shape Regression

  2015cited by this paper
• Facial Landmark Detection via Progressive Initialization

  2015cited by this paper
• Facial Landmark Tracking by Tree-Based Deformable Part Model Based Detector

  2015cited by this paper
• Automatic Construction of Deformable Models In-the-Wild

  2014cited by this paper
• DeepFace: Closing the Gap to Human-Level Performance in Face Verification

  2014cited by this paper
• Incremental Face Alignment in the Wild

  2014influential reference
• Fast Newton active appearance models

  2014cited by this paper
• Gauss-Newton Deformable Part Models for Face Alignment In-the-Wild

  2014influential reference
• RAPS: Robust and Efficient Automatic Construction of Person-Specific Deformable Models

  2014cited by this paper
• Bayesian Active Appearance Models

  2014cited by this paper
• Active Orientation Models for Face Alignment In-the-Wild

  2014cited by this paper
• Menpo: A Comprehensive Platform for Parametric Image Alignment and Visual Deformable Models

  2014cited by this paper
• Face Alignment at 3000 FPS via Regressing Local Binary Features

  2014cited by this paper
• Localizing Parts of Faces Using a Consensus of Exemplars

  2013cited by this paper
• UvA-DARE ( Digital Academic Repository ) Visual Tracking : An Experimental Survey Smeulders

  2013cited by this paper
• 300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge

  2013influential reference
• Hierarchical On-line Appearance-Based Tracking for 3D head pose, eyebrows, lips, eyelids and irises

  2013cited by this paper
• Optimization Problems for Fast AAM Fitting in-the-Wild

  2013cited by this paper
• Supervised Descent Method and Its Applications to Face Alignment

  2013influential reference
• Efficient Online Subspace Learning With an Indefinite Kernel for Visual Tracking and Recognition

  2012cited by this paper
• Face detection, pose estimation, and landmark localization in the wild

  2012cited by this paper
• Interactive Facial Feature Localization

  2012cited by this paper
• Euler Principal Component Analysis

  2012cited by this paper
• Real-Time Facial Feature Tracking on a Mobile Device

  2012cited by this paper
• Annotated Facial Landmarks in the Wild: A large-scale, real-world database for facial landmark localization

  2011cited by this paper
• A Dynamic Texture-Based Approach to Recognition of Facial Actions and Their Temporal Models

  2010cited by this paper
• Face-TLD: Tracking-Learning-Detection applied to faces

  2010cited by this paper
• On compositional Image Alignment, with an application to Active Appearance Models

  2009cited by this paper
• Face tracking and recognition with visual constraints in real-world videos

  2008cited by this paper
• Adaptive and constrained algorithms for inverse compositional Active Appearance Model fitting

  2008cited by this paper
• Incremental Learning for Robust Visual Tracking

  2008cited by this paper
• Multi-PIE

  2008cited by this paper
• IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE , VOL . ? , NO . ? , ? 2007 1 Image Stitching Using Structure Deformation

  2007cited by this paper
• IEEE transactions on speech and audio processing : a publication of the IEEE Signal Processing Society.

  2005cited by this paper
• Real-time combined 2D+3D active appearance models

  2004cited by this paper
• The template update problem

  2004cited by this paper
• Particle filtering with factorized likelihoods for tracking facial features

  2004cited by this paper
• Active Appearance Models Revisited

  2004cited by this paper
• Robust online appearance models for visual tracking

  2001cited by this paper
• Robust Real-Time Face Detection

  2001cited by this paper
• Optical Flow Constraints on Deformable Models with Applications to Face Tracking

  2000cited by this paper
• LAFTER: lips and face real time tracker

  1997cited by this paper
• Face localization and facial feature extraction based on shape and color information

  1996cited by this paper
• Recognizing Human Facial Expressions From Long Image Sequences Using Optical Flow

  1996cited by this paper
• Modeling, tracking and interactive animation of faces and heads//using input from video

  1996cited by this paper
• Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion

  1995cited by this paper
• A unified approach to coding and interpreting face images

  1995cited by this paper
• This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON AFFECTIVE COMPUTING

  year unknowncited by this paper

• Supervision-by-Hallucination-and-Transfer: A Weakly-Supervised Approach for Robust and Precise Facial Landmark Detection

  2026influential citation
• GIFT: Generated Indoor video frames for Texture-less point tracking

  2025cites this paper
• Semi-supervised denoising for facial landmark detection in videos

  2025cites this paper
• Unsupervised Discovery of Facial Landmarks and Head Pose

  2025cites this paper
• Learning Person-Specific Animatable Face Models from In-the-Wild Images via a Shared Base Model

  2025cites this paper
• Facial video semantic coding for semantic communication

  2025cites this paper
• Facial Animation Retargeting by Unsupervised Learning of Graph Convolutional Networks

  2024cites this paper
• Evaluation of Head Pose Estimation Algorithms for Sign Language Analysis

  2024cites this paper
• A Study Of Methods For Detecting Facial Occlusion Based On Template Matching

  2024cites this paper
• Learning from Machine Learning: advancing from static to dynamic facial function quantification

  2024cites this paper
• One-Shot Neural Face Reenactment via Finding Directions in GAN’s Latent Space

  2024cites this paper
• FaceXFormer: A Unified Transformer for Facial Analysis

  2024cites this paper
• TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement

  2023cites this paper
• Modular Multi-Face Tracking Geared Toward Face Recognition in Surveillance Videos

  2023cites this paper
• Video-Based Stabilized 3D Face Alignment Using Temporal Multi-Discrimination

  2023cites this paper
• Learning One-Quarter Headshot 3D GANs from a Single-View Portrait Dataset with Diverse Body Poses

  2023cites this paper
• AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image Collections

  2023cites this paper
• Deepfakes Generation and Detection: A Short Survey

  2023cites this paper
• A store-and-forward cloud-based telemonitoring system for automatic assessing dysarthria evolution in neurological diseases from video-recording analysis

  2023cites this paper
• 3D-aware Facial Landmark Detection via Multi-view Consistent Training on Synthetic Data

  2023cites this paper
• 1DFormer: Learning 1D Landmark Representations via Transformer for Facial Landmark Tracking

  2023cites this paper
• Stack of Heatmap Hourglass Network for Face Alignment

  2023cites this paper
• STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detection

  2023cites this paper
• Disentangled representation transformer network for 3D face reconstruction and robust dense alignment

  2023cites this paper
• 3DPortraitGAN: Learning One-Quarter Headshot 3D GANs From a Single-View Portrait Dataset With Diverse Body Poses

  2023cites this paper
• 1DFormer: A Transformer Architecture Learning 1D Landmark Representations for Facial Landmark Tracking

  2023cites this paper
• Stabilized Temporal 3D Face Alignment Using Landmark Displacement Learning

  2023cites this paper
• Face Reenactment with Diffusion Model and Its Application to Video Compression

  2023cites this paper
• RULe: Relocalization-Uniformization-Landmark Estimation Network for Real-Time Face Alignment in Degraded Conditions

  2023cites this paper
• Recurrence without Recurrence: Stable Video Landmark Detection with Deep Equilibrium Models

  2023cites this paper
• LocalEyenet: Deep Attention framework for Localization of Eyes

  2023cites this paper
• Learning Full-Head 3D GANs from a Single-View Portrait Dataset

  2023cites this paper
• Toward Robust Facial Action Units’ Detection

  2023cites this paper
• LPFF: A Portrait Dataset for Face Generators Across Large Poses

  2023cites this paper
• Multi-Label Compound Expression Recognition: C-EXPR Database & Network

  2023cites this paper
• Dual Learning for Joint Facial Landmark Detection and Action Unit Recognition

  2023cites this paper
• HyperReenact: One-Shot Reenactment via Jointly Learning to Refine and Retarget Faces

  2023cites this paper
• The Heat is On: Thermal Facial Landmark Tracking

  2023cites this paper
• Driver-Drowsiness Detection System Using Deep Learning (CNN)

  2023influential citation
• Driver Take-Over Behaviour Study Based on Gaze Focalization and Vehicle Data in CARLA Simulator

  2022cites this paper
• TAP-Vid: A Benchmark for Tracking Any Point in a Video

  2022cites this paper
• WATB: Wild Animal Tracking Benchmark

  2022cites this paper
• On-Device CPU Scheduling for Sense-React Systems

  2022cites this paper
• Evaluation of Various State of the Art Head Pose Estimation Algorithms for Clinical Scenarios

  2022cites this paper
• MHR-Net: Multiple-Hypothesis Reconstruction of Non-Rigid Shapes from 2D Views

  2022cites this paper
• Continuous Facial Motion Deblurring

  2022influential citation
• From Keypoints to Object Landmarks via Self-Training Correspondence: A Novel Approach to Unsupervised Landmark Discovery

  2022cites this paper
• TransMarker: A Pure Vision Transformer for Facial Landmark Detection

  2022influential citation
• Consensus and stacking based fusion and survey of facial feature point detectors

  2022cites this paper
• ARCosmetics: a real-time augmented reality cosmetics try-on system

  2022cites this paper
• Optimizable face detection and tracking model with occlusion resolution for high quality videos

  2022cites this paper
• Finding Directions in GAN's Latent Space for Neural Face Reenactment

  2022cites this paper
• Improving face recognition by pose-aware quality assessment and judgement

  2022cites this paper
• Challenges and Opportunities for Machine Learning Classification of Behavior and Mental State from Images

  2022cites this paper
• Adversarial Stacking Ensemble for Facial Landmark Tracking

  2022cites this paper
• Pose for Everything: Towards Category-Agnostic Pose Estimation

  2022cites this paper
• Correlation filter tracking based on superpixel and multifeature fusion

  2021cites this paper
• Fast Facial Landmark Detection and Applications: A Survey

  2021cites this paper
• Weakly-Supervised Multi-Face 3D Reconstruction

  2021cites this paper
• F³A-GAN: Facial Flow for Face Animation With Generative Adversarial Networks

  2021cites this paper
• Towards Unconstrained Facial Landmark Detection Robust to Diverse Cropping Manners

  2021cites this paper
• Regression-based methods for face alignment: A survey

  2021cites this paper
• Rethinking Eye-blink: Assessing Task Difficulty through Physiological Representation of Spontaneous Blinking

  2021cites this paper
• Face Tracking via Content Aware Correlation Filter

  2021cites this paper
• Improving Face Alignment Accuracy on Clinical Populations and its effect on the Video-based Detection of Neurological Diseases

  2021influential citation
• Subpixel Heatmap Regression for Facial Landmark Localization

  2021cites this paper
• Reasoning structural relation for occlusion-robust facial landmark localization

  2021influential citation
• Improving Robustness of Facial Landmark Detection by Defending against Adversarial Attacks

  2021cites this paper
• Synergy between 3DMM and 3D Landmarks for Accurate 3D Facial Geometry

  2021cites this paper
• Disentangled Representation Learning and Its Application to Face Analytics

  2021cites this paper
• Face Restoration via Plug-and-Play 3D Facial Priors

  2021cites this paper
• Gaze Focalization System for Driving Applications Using OpenFace 2.0 Toolkit with NARMAX Algorithm in Accidental Scenarios

  2021cites this paper
• Learning to Deblur and Rotate Motion-Blurred Faces

  2021cites this paper
• Composite recurrent network with internal denoising for facial alignment in still and video images in the wild

  2021influential citation
• Mask6D: Masked Pose Priors for 6D Object Pose Estimation

  2021cites this paper
• Real-time face alignment: evaluation methods, training strategies and implementation optimization

  2021cites this paper
• The DEVIL is in the Details: A Diagnostic Evaluation Benchmark for Video Inpainting

  2021cites this paper
• Breaking Shortcut: Exploring Fully Convolutional Cycle-Consistency for Video Correspondence Learning

  2021influential citation
• Design of a Facial Landmark Detection System Using a Dynamic Optical Flow Approach

  2021cites this paper
• Inverting Generative Adversarial Renderer for Face Reconstruction

  2021cites this paper
• Deep Face Video Inpainting via UV Mapping

  2021cites this paper
• A Brief Review of 3D Face Reconstruction Methods for Face-Related Product Design

  2021cites this paper
• Development of the Wits Face Database: an African database of high-resolution facial photographs and multimodal closed-circuit television (CCTV) recordings

  2021cites this paper
• 3-D Facial Landmarks Detection for Intelligent Video Systems

  2021cites this paper
• FineNet: Frame Interpolation and Enhancement for Face Video Deblurring

  2021cites this paper
• An Efficient Multitask Neural Network for Face Alignment, Head Pose Estimation and Face Tracking

  2021cites this paper
• Py-Feat: Python Facial Expression Analysis Toolbox

  2021cites this paper
• A Multi-Stage, Multi-Feature Machine Learning Approach to Detect Driver Sleepiness in Naturalistic Road Driving Conditions

  2021cites this paper
• Accurate 3D Facial Geometry Prediction by Multi-Task, Multi-Modal, and Multi-Representation Landmark Refinement Network

  2021cites this paper
• Robust and Precise Facial Landmark Detection by Self-Calibrated Pose Attention Network

  2021cites this paper
• Calibrated Multivariate Regression Networks

  2020cites this paper
• Multi-Task Head Pose Estimation in-the-Wild

  2020cites this paper
• High-Fidelity Monocular Face Reconstruction Based on an Unsupervised Model-Based Face Autoencoder

  2020cites this paper
• A Backbone Replaceable Fine-tuning Framework for Stable Face Alignment

  2020cites this paper
• Teacher-Student Asynchronous Learning with Multi-Source Consistency for Facial Landmark Detection

  2020cites this paper
• Geometry-attentive relational reasoning for robust facial landmark detection

  2020cites this paper
• The Effect of Improving Facial Alignment Accuracy on the Video-based Detection of Neurological Diseases

  2020influential citation
• Integrating OpenFace 2.0 Toolkit for Driver Attention Estimation in Challenging Accidental Scenarios

  2020cites this paper
• Spatial-Temporal Knowledge Integration: Robust Self-Supervised Facial Landmark Tracking

  2020cites this paper
• Efficient deformable 3D face model tracking with limited hardware resources

  2020influential citation