How Well Do Distributional Models Capture Different Types of Semantic Knowledge?

In recent years, distributional models (DMs) have shown great success in representing lexical semantics. In this work we show that the extent to which DMs represent semantic knowledge is highly dependent on the type of knowledge. We pose the task of predicting properties of concrete nouns in a supervised setting, and compare between learning taxonomic properties (e.g., animacy) and attributive properties (e.g., size, color). We employ four state-of-the-art DMs as sources of feature representation for this task, and show that they all yield poor results when tested on attributive properties, achieving no more than an average F-score of 0.37 in the binary property prediction task, compared to 0.73 on taxonomic properties. Our results suggest that the distributional hypothesis may not be equally applicable to all types of semantic information.

• Publication year

  2015

• Venue

  Annual Meeting of the Association for Computational Linguistics

• Publication date

  2015-07-01

• Fields of study

  Linguistics, Computer Science

• Identifiers
  DOI 10.3115/v1/P15-2119
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors

  2014cited by this paper
• Automatic extraction of property norm-like data from large text corpora

  2014cited by this paper
• GloVe: Global Vectors for Word Representation

  2014cited by this paper
• Dependency-Based Word Embeddings

  2014cited by this paper
• Semantic Frame Identification with Distributed Word Representations

  2014cited by this paper
• SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation

  2014cited by this paper
• Minimally Supervised Classification to Semantic Categories using Automatically Acquired Symmetric Patterns

  2014cited by this paper
• Fisher Vectors Derived from Hybrid Gaussian-Laplacian Mixture Models for Image Annotation

  2014cited by this paper
• Parsing with Compositional Vector Grammars

  2013cited by this paper
• Distributed Representations of Words and Phrases and their Compositionality

  2013cited by this paper
• Perceptual Inference Through Global Lexical Similarity

  2012cited by this paper
• LIBSVM: A library for support vector machines

  2011cited by this paper
• Symbol Interdependency in Symbolic and Embodied Cognition

  2011cited by this paper
• Distributional Memory: A General Framework for Corpus-Based Semantics

  2010cited by this paper
• Strudel: A Corpus-Based Semantic Model Based on Properties and Types

  2010cited by this paper
• Integrating experiential and distributional data to learn semantic representations.

  2009cited by this paper
• Towards Unrestricted, Large-Scale Acquisition of Feature-Based Conceptual Representations from Corpus Data

  2009cited by this paper
• Concepts and properties in word spaces

  2008influential reference
• Semantic feature production norms for a large set of living and nonliving things

  2005cited by this paper
• Concept Learning and Categorization from the Web

  2005cited by this paper
• Distributional Structure

  1954cited by this paper

• Seeing What Tastes Good: Revisiting Multimodal Distributional Semantics in the Billion Parameter Era

  2025cites this paper
• Supporting Systems Modeling Using Semantic Cues for Concept Development

  2025cites this paper
• Visual experience modulates the sensitivity to the distributional history of words in natural language

  2024cites this paper
• Ranking Entities along Conceptual Space Dimensions with LLMs: An Analysis of Fine-Tuning Strategies

  2024cites this paper
• Encoding and Decoding Graph Representations of Natural Language

  2024cites this paper
• What do Deck Chairs and Sun Hats Have in Common? Uncovering Shared Properties in Large Concept Vocabularies

  2023cites this paper
• Semantic features of object concepts generated with GPT-3

  2022cites this paper
• Interpretation of Black Box NLP Models: A Survey

  2022cites this paper
• COMPS: Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Models

  2022cites this paper
• COMPS: Conceptual Minimal Pair Sentences for testing Property Knowledge and Inheritance in Pre-trained Language Models

  2022cites this paper
• Beyond the Benchmarks: Toward Human-Like Lexical Representations

  2022cites this paper
• Assessing the Limits of the Distributional Hypothesis in Semantic Spaces: Trait-based Relational Knowledge and the Impact of Co-occurrences

  2022cites this paper
• On the Limitations of Dataset Balancing: The Lost Battle Against Spurious Correlations

  2022cites this paper
• Language in a (Search) Box: Grounding Language Learning in Real-World Human-Machine Interaction

  2021cites this paper
• Unsupervised Grammatical Pattern Discovery from Arabic Extra Large Corpora

  2021cites this paper
• ALL Dolphins Are Intelligent and SOME Are Friendly: Probing BERT for Nouns’ Semantic Properties and their Prototypicality

  2021cites this paper
• Method of Improving Semantic Representation of Word Embedding

  2021cites this paper
• Grounding semantic transparency in context

  2021cites this paper
• Do Language Models Know the Way to Rome?

  2021cites this paper
• Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color

  2021cites this paper
• Shared understanding of color among sighted and blind adults

  2021cites this paper
• A Formidable Ability: Detecting Adjectival Extremeness with DSMs

  2021cites this paper
• Representation and Pre-Activation of Lexical-Semantic Knowledge in Neural Language Models

  2021cites this paper
• Familiar words but strange voices: Modelling the influence of speech variability on word recognition

  2021cites this paper
• Local Interpretations for Explainable Natural Language Processing: A Survey

  2021cites this paper
• Pre-trained models for natural language processing: A survey

  2020cites this paper
• Interpreting Sequence-to-Sequence Models for Russian Inflectional Morphology

  2020cites this paper
• On the Existence of Tacit Assumptions in Contextualized Language Models

  2020cites this paper
• Predicting Declension Class from Form and Meaning

  2020cites this paper
• Probing Neural Language Models for Human Tacit Assumptions

  2020cites this paper
• Why is penguin more similar to polar bear than to sea gull? Analyzing conceptual knowledge in distributional models

  2020cites this paper
• Semantic memory: A review of methods, models, and current challenges

  2020cites this paper
• A university map of course knowledge

  2020cites this paper
• Analysing Word Representation from the Input and Output Embeddings in Neural Network Language Models

  2020cites this paper
• Encoding Lexico-Semantic Knowledge using Ensembles of Feature Maps from Deep Convolutional Neural Networks

  2020cites this paper
• Mental representations of objects reflect the ways in which we interact with them

  2020cites this paper
• Shared understanding of color among congenitally blind and sighted adults

  2020cites this paper
• U NDERSTANDING M ENTAL R EPRESENTATIONS O F O BJECTS T HROUGH V ERBS A PPLIED T O T HEM

  2020cites this paper
• Understanding the role of linguistic distributional knowledge in cognition

  2019cites this paper
• Deconstructing Word Embeddings

  2019cites this paper
• Feature2Vec: Distributional semantic modelling of human property knowledge

  2019cites this paper
• Do Neural Language Representations Learn Physical Commonsense?

  2019cites this paper
• Topic Embeddings - A New Approach to Classify Very Short Documents Based on Predefined Topics

  2019cites this paper
• A Systematic Comparison of English Noun Compound Representations

  2019cites this paper
• Mapping Distributional Semantics to Property Norms with Deep Neural Networks

  2019cites this paper
• Relational Word Embeddings

  2019cites this paper
• Knowledge of animal appearance among sighted and blind adults

  2019cites this paper
• Deconstructing Word Embeddings

  2019cites this paper
• Inducing Relational Knowledge from BERT

  2019cites this paper
• Reply to Lewis et al.: Inference is key to learning appearance from language, for humans and distributional semantic models alike

  2019cites this paper
• Traces of Meaning Itself: Encoding Distributional Word Vectors in Brain Activity

  2019cites this paper
• Learning Joint Gaussian Representations for Movies, Actors, and Literary Characters

  2018cites this paper
• Semantic projection: recovering human knowledge of multiple, distinct object features from word embeddings

  2018cites this paper
• Joint Sentiment Topic Model for objective text clustering

  2018cites this paper
• Understanding Actors and Evaluating Personae with Gaussian Embeddings

  2018cites this paper
• A Genetic Algorithm for Combining Visual and Textual Embeddings Evaluated on Attribute Recognition

  2018cites this paper
• Evaluating Word Embeddings in Multi-label Classification Using Fine-Grained Name Typing

  2018cites this paper
• NTU NLP Lab System at SemEval-2018 Task 10: Verifying Semantic Differences by Integrating Distributional Information and Expert Knowledge

  2018cites this paper
• Bridging CNNs, RNNs, and Weighted Finite-State Machines

  2018cites this paper
• Investigating Inner Properties of Multimodal Representation and Semantic Compositionality with Brain-based Componential Semantics

  2017cites this paper
• Distributed representations for fine-grained entity typing

  2017cites this paper
• Zero-Shot Activity Recognition with Verb Attribute Induction

  2017cites this paper
• Pattern-based methods for Improved Lexical Semantics and Word Embeddings

  2017cites this paper
• Is this a Child, a Girl or a Car? Exploring the Contribution of Distributional Similarity to Learning Referential Word Meanings

  2017cites this paper
• Distributional model on a diet: One-shot word learning from text only

  2017cites this paper
• Towards Interpretable Vision Systems

  2017cites this paper
• Distributional Modeling on a Diet: One-shot Word Learning from Text Only

  2017cites this paper
• Verb Physics: Relative Physical Knowledge of Actions and Objects

  2017cites this paper
• Argumentation mining: How can a machine acquire common sense and world knowledge?

  2017cites this paper
• Are Distributional Representations Ready for the Real World? Evaluating Word Vectors for Grounded Perceptual Meaning

  2017cites this paper
• Sentence level matrix representation for document spectral clustering

  2017cites this paper
• Modeling Context Words as Regions: An Ordinal Regression Approach to Word Embedding

  2017cites this paper
• Using Continuous Representation of Various Linguistic Units for Recurrent Neural Network based TTS Synthesis

  2016cites this paper
• Intrinsic Subspace Evaluation of Word Embedding Representations

  2016cites this paper
• HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment

  2016cites this paper
• Investigation of Using Continuous Representation of Various Linguistic Units in Neural Network Based Text-to-Speech Synthesis

  2016cites this paper
• PAPER Special Section on Recent Advances in Machine Learning for Spoken Language Processing Investigation of Using Continuous Representation of Various Linguistic Units in Neural Network Based Text-to-Speech Synthesis

  2016cites this paper
• Is an Image Worth More than a Thousand Words? On the Fine-Grain Semantic Differences between Visual and Linguistic Representations

  2016influential citation
• Symmetric Patterns and Coordinations: Fast and Enhanced Representations of Verbs and Adjectives

  2016cites this paper
• Enhance the Word Vector with Prosodic Information for the Recurrent Neural Network Based TTS System

  2016cites this paper
• Distributional vectors encode referential attributes

  2015cites this paper