Continuous Conditional Neural Fields for Structured Regression

An increasing number of computer vision and pattern recognition problems require structured regression techniques. Problems like human pose estimation, unsegmented action recognition, emotion prediction and facial landmark detection have temporal or spatial output dependencies that regular regression techniques do not capture. In this paper we present continuous conditional neural fields (CCNF) – a novel structured regression model that can learn non-linear input-output dependencies, and model temporal and spatial output relationships of varying length sequences. We propose two instances of our CCNF framework: Chain-CCNF for time series modelling, and Grid-CCNF for spatial relationship modelling. We evaluate our model on five public datasets spanning three different regression problems: facial landmark detection in the wild, emotion prediction in music and facial action unit recognition. Our CCNF model demonstrates state-of-the-art performance on all of the datasets used.

• Publication year

  2014

• Venue

  European Conference on Computer Vision

• Publication date

  2014-09-06

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1007/978-3-319-10593-2_39
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Robust Discriminative Response Map Fitting with Constrained Local Models

  2013influential reference
• What Really Matters? A Study into People's Instinctive Evaluation Metrics for Continuous Emotion Prediction in Music

  2013cited by this paper
• Markov Random Field Structures for Facial Action Unit Intensity Estimation

  2013cited by this paper
• Dimensional affect recognition using Continuous Conditional Random Fields

  2013influential reference
• DISFA: A Spontaneous Facial Action Intensity Database

  2013cited by this paper
• Continuous AU intensity estimation using localized, sparse facial feature space

  2013cited by this paper
• Supervised Descent Method and Its Applications to Face Alignment

  2013cited by this paper
• 300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge

  2013cited by this paper
• AVEC 2013: the continuous audio/visual emotion and depression recognition challenge

  2013cited by this paper
• Constrained Local Neural Fields for Robust Facial Landmark Detection in the Wild

  2013cited by this paper
• Localizing Parts of Faces Using a Consensus of Exemplars

  2013influential reference
• Emotion tracking in music using continuous conditional random fields and relative feature representation

  2013cited by this paper
• Face detection, pose estimation, and landmark localization in the wild

  2012influential reference
• Discriminative Bayesian Active Shape Models

  2012cited by this paper
• Interactive Facial Feature Localization

  2012influential reference
• AVEC 2012: the continuous audio/visual emotion challenge

  2012cited by this paper
• Segmentation Propagation in ImageNet

  2012cited by this paper
• Output-associative RVM regression for dimensional and continuous emotion prediction

  2011cited by this paper
• A Comparative Study of Collaborative vs. Traditional Musical Mood Annotation

  2011influential reference
• Relevance Vector Machine Based Speech Emotion Recognition

  2011cited by this paper
• Continuous Conditional Random Fields for Regression in Remote Sensing

  2010cited by this paper
• Twin Gaussian Processes for Structured Prediction

  2010cited by this paper
• Deformable Model Fitting by Regularized Landmark Mean-Shift

  2010influential reference
• Conditional Neural Fields

  2009cited by this paper
• Structured output-associative regression

  2009cited by this paper
• SMERS: Music Emotion Recognition Using Support Vector Regression

  2009cited by this paper
• Multi-PIE

  2008influential reference
• Enforcing convexity for improved alignment with constrained local models

  2008cited by this paper
• Toward Faster Nonnegative Matrix Factorization: A New Algorithm and Comparisons

  2008cited by this paper
• LIBLINEAR: A Library for Large Linear Classification

  2008cited by this paper
• Global Ranking Using Continuous Conditional Random Fields

  2008cited by this paper
• Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors

  2008cited by this paper
• Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments

  2008cited by this paper
• An Introduction to Conditional Random Fields for Relational Learning

  2007cited by this paper
• Pattern Recognition and Machine Learning

  2006influential reference
• Support vector machine learning for interdependent and structured output spaces

  2004cited by this paper
• Active Appearance Models Revisited

  2004cited by this paper
• A Limited Memory Algorithm for Bound Constrained Optimization

  1995cited by this paper

• Confidence-Aware 3D Gaze Tracking and Assessment Metric

  2025cites this paper
• Weak and Wasserstein convergence of periodic measures of stochastic neural field lattice models with Heaviside 's operators and locally Lipschitz Lévy noises

  2025cites this paper
• Efficient End-to-End Convolutional Architecture for Point-of-Gaze Estimation

  2024cites this paper
• Confidence-aware 3D Gaze Estimation and Evaluation Metric

  2023cites this paper
• Unconstrained human gaze estimation approach for medium-distance scene based on monocular vision

  2023cites this paper
• Ice Thickness From Deep Learning and Conditional Random Fields: Application to Ice-Penetrating Radar Data With Radiometric Validation

  2022cites this paper
• Consensus and stacking based fusion and survey of facial feature point detectors

  2022influential citation
• Gaze Estimation via a Differential Eyes’ Appearances Network with a Reference Grid

  2021cites this paper
• LNSMM: Eye Gaze Estimation With Local Network Share Multiview Multitask

  2021cites this paper
• Auxiliary diagnostic system for ADHD in children based on AI technology

  2021cites this paper
• Effect Of Personalized Calibration On Gaze Estimation Using Deep-Learning

  2021cites this paper
• Gaze Tracking Using an Unmodified Web Camera and Convolutional Neural Network

  2021cites this paper
• ACN: Occlusion-tolerant face alignment by attentional combination of heterogeneous regression networks

  2020cites this paper
• Unsupervised Learning Facial Parameter Regressor for Action Unit Intensity Estimation via Differentiable Renderer

  2020cites this paper
• AnchorFace: An Anchor-based Facial Landmark Detector Across Large Poses

  2020cites this paper
• Vigilance Estimation Using a Wearable EOG Device in Real Driving Environment

  2020cites this paper
• Neural Probabilistic Graphical Model for Face Sketch Synthesis

  2020cites this paper
• Deep Structured Prediction for Facial Landmark Detection

  2020cites this paper
• Robust Facial Landmark Detection via Heatmap-Offset Regression

  2020cites this paper
• A complementary regression network for accurate face alignment

  2020cites this paper
• Efficiency in Real-time Webcam Gaze Tracking

  2020cites this paper
• An Accurate and Robust Gaze Estimation Method Based on Maximum Correntropy Criterion

  2019cites this paper
• PFLD: A Practical Facial Landmark Detector

  2019cites this paper
• Effective approach for facial expression recognition using hybrid square-based diagonal pattern geometric model

  2019cites this paper
• The ASC-Inclusion Perceptual Serious Gaming Platform for Autistic Children

  2019cites this paper
• Practical Approaches towards Complete Real-time Gaze Tracking

  2019cites this paper
• Feature extraction on faces : from landmark localization to depth estimation

  2019cites this paper
• Improved Search Strategy for Active Shape Model of Face Alignment

  2019cites this paper
• Distance-To-Mean Continuous Conditional Random Fields: Case Study in Traffic Congestion

  2019cites this paper
• Learning Robust Facial Landmark Detection via Hierarchical Structured Ensemble

  2019cites this paper
• Context-Aware Feature and Label Fusion for Facial Action Unit Intensity Estimation With Partially Labeled Data

  2019cites this paper
• Semi-Supervised Deep Neural Network for Joint Intensity Estimation of Multiple Facial Action Units

  2019cites this paper
• Neural Network based Continuous Conditional Random Field for Fine-grained Crime Prediction

  2019cites this paper
• Stacked Multi-Target Network for Robust Facial Landmark Localisation

  2019cites this paper
• Temporal Graph Regression via Structure-Aware Intrinsic Representation Learning

  2019cites this paper
• Facial landmark localization by enhanced convolutional neural network

  2018cites this paper
• OpenFace 2.0: Facial Behavior Analysis Toolkit

  2018cites this paper
• Transductive Regression for Data With Latent Dependence Structure

  2018cites this paper
• Multimodal Depression Detection: Fusion Analysis of Paralinguistic, Head Pose and Eye Gaze Behaviors

  2018cites this paper
• Accurate face alignment and adaptive patch selection for heart rate estimation from videos under realistic scenarios

  2018cites this paper
• Markov Random Neural Fields for Face Sketch Synthesis

  2018cites this paper
• A comparison of facial landmark detection methods

  2018cites this paper
• Bilateral Ordinal Relevance Multi-instance Regression for Facial Action Unit Intensity Estimation

  2018cites this paper
• Robust Facial Landmark Detection via a Fully-Convolutional Local-Global Context Network

  2018influential citation
• Multimodal Emotion Recognition using Deep Continuous Conditional Recurrent Neural Fields

  2018cites this paper
• Weakly-Supervised Deep Convolutional Neural Network Learning for Facial Action Unit Intensity Estimation

  2018cites this paper
• Edge Convolutional Network for Facial Action Intensity Estimation

  2018cites this paper
• Gaze estimation and interaction in real-world environments

  2018influential citation
• Recognizing Emotions in Video Using Multimodal DNN Feature Fusion

  2018cites this paper
• Facial Landmark Detection: A Literature Survey

  2018cites this paper
• Preliminary research on continuous conditional random fields in predicting high-dimensional data

  2017cites this paper
• Combining Sequential Geometry and Texture Features for Distinguishing Genuine and Deceptive Emotions

  2017cites this paper
• Local-global ranking for facial expression intensity estimation

  2017cites this paper
• Single image depth prediction using super-column super-pixel features

  2017cites this paper
• Joint Face Detection and Initialization for Face Alignment

  2017cites this paper
• Convolutional Experts Constrained Local Model for 3D Facial Landmark Detection

  2017cites this paper
• Continuous Conditional Dependency Network for Structured Regression

  2017cites this paper
• Distance-to-Mean Continuous Conditional Random Fields to Enhance Prediction Problem in Traffic Flow Data

  2017cites this paper
• Just‐in‐time, viable, 3‐D avatars from scans

  2017cites this paper
• Local-Global Landmark Confidences for Face Recognition

  2017influential citation
• Spatiotemporal Dynamics of Neural Signals : Modeling and Inference

  2017influential citation
• Bio-acoustic emotion recognition using continuous conditional recurrent neural fields

  2017cites this paper
• MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation

  2017influential citation
• Detecting Gaze Towards Eyes in Natural Social Interactions and Its Use in Child Assessment

  2017cites this paper
• Probabilistic graphical models for credibility analysis in evolving online communities

  2017cites this paper
• Constrained Ensemble Initialization for Facial Landmark Tracking in Video

  2017cites this paper
• MOOC-Gaze: Online Solution for Tracking Learners’ Gazing Dynamics of MOOC Videos

  2017cites this paper
• Robust statistical deformable models

  2017cites this paper
• A dense flow-based framework for real-time object registration under compound motion

  2017cites this paper
• MOOC-Gaze: Online Solution for Tracking Learners’ Gazing Dynamics of MOOC Videos

  2017cites this paper
• Convolutional Experts Constrained Local Model for Facial Landmark Detection

  2016influential citation
• Deep Constrained Local Models for Facial Landmark Detection

  2016cites this paper
• Hybrid Patch Based Diagonal Pattern Geometric Appearance Model for Facial Expression Recognition

  2016cites this paper
• Geometric Representations and Deep Gaussian Conditional Random field Networks for Computer Vision

  2016cites this paper
• Estimation du regard avec une caméra RGB-D dans des environnements utilisateur non-contraints

  2016cites this paper
• 1 . RELATED WORK 1 . 1 3 D Reconstruction

  2016influential citation
• Rapid Photorealistic Blendshape Modeling from RGB-D Sensors

  2016cites this paper
• Real-time facial action unit intensity prediction with regularized metric learning

  2016influential citation
• Robust face alignment and partial face recognition

  2016cites this paper
• Face alignment in-the-wild: A Survey

  2016cites this paper
• A multimodal approach to estimating vigilance using EEG and forehead EOG

  2016cites this paper
• Gaussian Conditional Random Field Network for Semantic Segmentation

  2016cites this paper
• Advances, Challenges, and Opportunities in Automatic Facial Expression Recognition

  2016cites this paper
• Neural conditional ordinal random fields for agreement level estimation

  2016cites this paper
• Pooling Faces: Template Based Face Recognition with Pooled Face Images

  2016cites this paper
• The Cambridge Face Tracker: Accurate, Low Cost Measurement of Head Posture Using Computer Vision and Face Recognition Software

  2016cites this paper
• Discriminative Training of Deep Fully Connected Continuous CRFs With Task-Specific Loss

  2016cites this paper
• Time-Delay Neural Network for Continuous Emotional Dimension Prediction From Facial Expression Sequences

  2016cites this paper
• Advances in compositional fitting of active appearance models

  2016influential citation
• It’s Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation

  2016cites this paper
• Holistically Constrained Local Model: Going Beyond Frontal Poses for Facial Landmark Detection

  2016influential citation
• MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos

  2016cites this paper
• List of code Yue Wu December 11 , 2016 1

  2016cites this paper
• AggreGaze: Collective Estimation of Audience Attention on Public Displays

  2016cites this paper
• Convolutional Experts Constrained Local Model for Facial Landmark Detection

  2016influential citation
• Automated recognition of complex categorical emotions from facial expressions and head motions

  2015cites this paper
• Measuring the intensity of spontaneous facial action units with dynamic Bayesian network

  2015cites this paper
• Continuous dimensional emotion tracking in music

  2015influential citation
• Regression-based estimation of pain and facial expression intensity

  2015influential citation
• Framework for combination aware AU intensity recognition

  2015cites this paper