Functionality Discovery and Prediction of Physical Objects

Functionality is a fundamental attribute of an object which indicates the capability to be used to perform specific actions. It is critical to empower robots the functionality knowledge in discovering appropriate objects for a task e.g. cut cake using knife. Existing research works have focused on understanding object functionality through human-object-interaction from extensively annotated image or video data and are hard to scale up. In this paper, we (1) mine object-functionality knowledge through pattern-based and model-based methods from text, (2) introduce a novel task on physical object-functionality prediction, which consumes an image and an action query to predict whether the object in the image can perform the action, and (3) propose a method to leverage the mined functionality knowledge for the new task. Our experimental results show the effectiveness of our methods.

• Publication year

  2020

• Venue

  AAAI Conference on Artificial Intelligence

• Publication date

  2020-04-03

• Fields of study

  Computer Science, Engineering

• Identifiers
  DOI 10.1609/AAAI.V34I01.5342
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Universal Dependency Parsing from Scratch

  2019cited by this paper
• Augmenting End-to-End Dialogue Systems With Commonsense Knowledge

  2018cited by this paper
• Estimation of Object Functions Using Visual Attention

  2018cited by this paper
• SimpleQuestions Nearly Solved: A New Upperbound and Baseline Approach

  2018cited by this paper
• Supervised Open Information Extraction

  2018influential reference
• R-VQA: Learning Visual Relation Facts with Semantic Attention for Visual Question Answering

  2018cited by this paper
• Embedding Uncertain Knowledge Graphs

  2018cited by this paper
• Stating the Obvious: Extracting Visual Common Sense Knowledge

  2016cited by this paper
• Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations

  2016cited by this paper
• Mining semantic affordances of visual object categories

  2015influential reference
• Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models

  2015cited by this paper
• Don't just listen, use your imagination: Leveraging visual common sense for non-visual tasks

  2015cited by this paper
• Learning Common Sense through Visual Abstraction

  2015cited by this paper
• Understanding tools: Task-oriented object modeling, learning and recognition

  2015cited by this paper
• Deep Residual Learning for Image Recognition

  2015cited by this paper
• Never-Ending Learning

  2015cited by this paper
• Reasoning about Object Affordances in a Knowledge Base Representation

  2014cited by this paper
• WebChild: harvesting and organizing commonsense knowledge from the web

  2014cited by this paper
• Embedding Entities and Relations for Learning and Inference in Knowledge Bases

  2014cited by this paper
• The Ecological Approach to Visual Perception: Classic Edition

  2014cited by this paper
• Discovering Object Functionality

  2013cited by this paper
• NEIL: Extracting Visual Knowledge from Web Data

  2013cited by this paper
• Translating Embeddings for Modeling Multi-relational Data

  2013cited by this paper
• Representing General Relational Knowledge in ConceptNet 5

  2012influential reference
• Syntactic Annotations for the Google Books NGram Corpus

  2012cited by this paper
• Random Walk Inference and Learning in A Large Scale Knowledge Base

  2011influential reference
• Function revisited: how infants construe functional features in their representation of objects.

  2008cited by this paper

• Mental representations of objects reflect the ways in which we interact with them

  2020cites this paper
• U NDERSTANDING M ENTAL R EPRESENTATIONS O F O BJECTS T HROUGH V ERBS A PPLIED T O T HEM

  2020cites this paper