Déjà Vu: - Motion Prediction in Static Images

This paper proposes motion prediction in single still images by learning it from a set of videos. The building assumption is that similar motion is characterized by similar appearance. The proposed method learns local motion patterns given a specific appearance and adds the predicted motion in a number of applications. This work (i) introduces a novel method to predict motion from appearance in a single static image, (ii) to that end, extends of the Structured Random Forest with regression derived from first principles, and (iii) shows the value of adding motion predictions in different tasks such as: weak frame-proposals containing unexpected events, action recognition, motion saliency. Illustrative results indicate that motion prediction is not only feasible, but also provides valuable information for a number of applications.

• Publication year

  2018

• Venue

  European Conference on Computer Vision

• Publication date

  2018-03-19

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1007/978-3-319-10578-9_12arXiv 1803.06951
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Filter Forests for Learning Data-Dependent Convolutional Kernels

  2014cited by this paper
• Structured Forests for Fast Edge Detection

  2013cited by this paper
• Decision Jungles: Compact and Rich Models for Classification

  2013cited by this paper
• Evaluation of Color STIPs for Human Action Recognition

  2013cited by this paper
• Motion Models that Only Work Sometimes

  2012cited by this paper
• Learning object class detectors from weakly annotated video

  2012cited by this paper
• Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning

  2012cited by this paper
• A Naturalistic Open Source Movie for Optical Flow Evaluation

  2012cited by this paper
• Structured class-labels in random forests for semantic image labelling

  2011influential reference
• Exploiting photographic style for category-level image classification by generalizing the spatial pyramid

  2011cited by this paper
• Recognizing human actions in still images: a study of bag-of-features and part-based representations

  2010influential reference
• Evaluating Color Descriptors for Object and Scene Recognition

  2010cited by this paper
• Episode-Constrained Cross-Validation in Video Concept Retrieval

  2009cited by this paper
• Learning actions from the Web

  2009cited by this paper
• Evaluation of Local Spatio-temporal Features for Action Recognition

  2009cited by this paper
• SIFT Flow: Dense Correspondence across Different Scenes

  2008cited by this paper
• Learning realistic human actions from movies

  2008cited by this paper
• Motion from blur

  2008cited by this paper
• A Spatio-Temporal Descriptor Based on 3D-Gradients

  2008cited by this paper
• Evaluation campaigns and TRECVid

  2006cited by this paper
• Human Detection Using Oriented Histograms of Flow and Appearance

  2006cited by this paper
• Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories

  2006cited by this paper
• Large Margin Methods for Structured and Interdependent Output Variables

  2005cited by this paper
• Recognizing human actions: a local SVM approach

  2004influential reference
• The Elements of Statistical Learning

  2003cited by this paper
• Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

  2001cited by this paper
• Random Forests

  2001cited by this paper
• Task assignment with unknown duration

  2000cited by this paper
• Visualizing 3D velocity fields near contour surfaces

  1994cited by this paper
• Imaging vector fields using line integral convolution

  1993cited by this paper
• Motion without movement

  1991cited by this paper

• Revisiting Feature Prediction for Learning Visual Representations from Video

  2024cites this paper
• Scrolly2Reel: Turning News Graphics into TikToks by Adjusting Narrative Beats and Pacing

  2024cites this paper
• Generative Image Dynamics

  2023cites this paper
• Objects do not disappear: Video object detection by single-frame object location anticipation

  2023cites this paper
• Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision

  2023cites this paper
• A Survey of Self-Supervised Learning from Multiple Perspectives: Algorithms, Theory, Applications and Future Trends

  2023cites this paper
• Investigation of Neural Network Algorithms for Human Movement Prediction Based on LSTM and Transformers

  2023cites this paper
• Video prediction for driving scenes with a memory differential motion network model

  2022cites this paper
• Hallucinating uncertain motion and future for static image action recognition

  2021cites this paper
• Dimensions of Motion: Monocular Prediction through Flow Subspaces

  2021cites this paper
• Self-supervised learning of Dynamic Representations for Static Images

  2021cites this paper
• Deep Gesture Video Generation With Learning on Regions of Interest

  2020cites this paper
• Adaptive Future Frame Prediction with Ensemble Network

  2020cites this paper
• Gait Recognition from a Single Image Using a Phase-Aware Gait Cycle Reconstruction Network

  2020cites this paper
• Instance-Level Future Motion Estimation in a Single Image Based on Ordinal Regression and Semi-Supervised Domain Adaptation

  2020cites this paper
• High-Quality Video Generation from Static Structural Annotations

  2020cites this paper
• Modeling urban growth using video prediction technology: A time‐dependent convolutional encoder–decoder architecture

  2020cites this paper
• Visual Reaction: Learning to Play Catch With Your Drone

  2019cites this paper
• Unsupervised Bi-directional Flow-based Video Generation from one Snapshot

  2019cites this paper
• Inferring Dynamic Representations of Facial Actions from a Still Image

  2019cites this paper
• DynamoNet: Dynamic Action and Motion Network

  2019cites this paper
• Cross-modality motion parameterization for fine-grained video prediction

  2019cites this paper
• Prediction of Sea Ice Motion With Convolutional Long Short-Term Memory Networks

  2019cites this paper
• Instance-Level Future Motion Estimation in a Single Image Based on Ordinal Regression

  2019cites this paper
• Exploiting Motion Information from Unlabeled Videos for Static Image Action Recognition

  2019cites this paper
• Modeling Structured Dynamics with Deep Neural Networks

  2019cites this paper
• A Comprehensive survey on deep future frame video prediction

  2018cites this paper
• Movement Prediction of Three Bouncing Balls

  2018cites this paper
• Dynamic Visual Sequence Prediction with Motion Flow Networks

  2018cites this paper
• Who Let the Dogs Out? Modeling Dog Behavior from Visual Data

  2018cites this paper
• Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networks

  2018cites this paper
• Video Time: Properties, Encoders and Evaluation

  2018cites this paper
• Data-Driven Visual Forecasting

  2018influential citation
• The Pose Knows: Video Forecasting by Generating Pose Futures

  2017cites this paper
• Hierarchically linked infinite hidden Markov model based trajectory analysis and semantic region retrieval in a trajectory dataset

  2017cites this paper
• Forecasting Human Dynamics from Static Images

  2017cites this paper
• Learning Object-Centric Transformation for Video Prediction

  2017cites this paper
• Activity Forecasting: An Invitation to Predictive Perception

  2017cites this paper
• Future-Supervised Retrieval of Unseen Queries for Live Video

  2017cites this paper
• Video Imagination from a Single Image with Transformation Generation

  2017cites this paper
• Exploiting visual motion to understand our visual world

  2017cites this paper
• Decomposing Motion and Content for Natural Video Sequence Prediction

  2017cites this paper
• Continuous learning in computer vision

  2017cites this paper
• "What Happens If..." Learning to Predict the Effect of Forces in Images

  2016cites this paper
• Predicting Future Optical Flow from Static Video Frames

  2016cites this paper
• Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks

  2016cites this paper
• One-Step Time-Dependent Future Video Frame Prediction with a Convolutional Encoder-Decoder Neural Network

  2016cites this paper
• Making a Case for Learning Motion Representations with Phase

  2016cites this paper
• An Uncertain Future: Forecasting from Static Images Using Variational Autoencoders

  2016cites this paper
• Newtonian Image Understanding: Unfolding the Dynamics of Objects in Static Images

  2015cites this paper
• Dense Optical Flow Prediction from a Static Image

  2015cites this paper
• RNN Encoder Decoder 3 D Skeleton Converter 2 D Pose Sequence Generator Input Image 2 D Pose Heatmaps 3 D Skeleton Hourglass network

  year unknowncites this paper