Word Embedding Calculus in Meaningful Ultradense Subspaces

We decompose a standard embedding space into interpretable orthogonal sub-spaces and a “remainder” subspace. We consider four interpretable subspaces in this paper: polarity, concreteness, frequency and part-of-speech (POS) sub-spaces. We introduce a new calculus for subspaces that supports operations like “ − 1 × hate = love ” and “give me a neutral word for greasy ” (i.e., oleaginous ). This calculus extends analogy computations like “ king − man + woman = queen ”. For the tasks of Antonym Classification and POS Tagging our method outperforms the state of the art. We create test sets for Morphological Analogies and for the new task of Polarity Spectrum Creation.

• Publication year

  2016

• Venue

  Annual Meeting of the Association for Computational Linguistics

• Publication date

  2016-08-01

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.18653/v1/P16-2083
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Morphological Word-Embeddings

  2019cited by this paper
• Ultradense Word Embeddings by Orthogonal Transformation

  2016influential reference
• Symmetric Pattern Based Word Embeddings for Improved Word Similarity Prediction

  2015cited by this paper
• Adjective Intensity and Sentiment Analysis

  2015cited by this paper
• Online Updating of Word Representations for Part-of-Speech Tagging

  2015cited by this paper
• Using Mined Coreference Chains as a Resource for a Semantic Task

  2014cited by this paper
• Compositional Morphology for Word Representations and Language Modelling

  2014cited by this paper
• FLORS: Fast and Simple Domain Adaptation for Part-of-Speech Tagging

  2014influential reference
• Concreteness ratings for 40 thousand generally known English word lemmas

  2014cited by this paper
• Efficient Estimation of Word Representations in Vector Space

  2013cited by this paper
• Good, Great, Excellent: Global Inference of Semantic Intensities

  2013cited by this paper
• CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON

  2013cited by this paper
• Polarity Inducing Latent Semantic Analysis

  2012cited by this paper
• Distance Metric Learning for Large Margin Nearest Neighbor Classification

  2005cited by this paper
• Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis

  2005cited by this paper
• Mining and summarizing customer reviews

  2004cited by this paper
• Book Reviews: WordNet: An Electronic Lexical Database

  1999cited by this paper

• Uncovering patterns of semantic predictability in sentence processing

  2025cites this paper
• Explainable text representation method with a learnable and explicit semantic space

  2024cites this paper
• Modelling Commonsense Commonalities with Multi-Facet Concept Embeddings

  2024cites this paper
• Word Embeddings Are Steers for Language Models

  2023cites this paper
• Dialectograms: Machine Learning Differences between Discursive Communities

  2023cites this paper
• Finding Interpretable Word Embedding Subspaces using Covariance and Correlation Maximization

  2022cites this paper
• SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Features

  2022cites this paper
• Interpreting Embedding Spaces by Conceptualization

  2022cites this paper
• On the Explainability of Natural Language Processing Deep Models

  2022influential citation
• SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable AMR Meaning Features

  2022cites this paper
• Learning interpretable word embeddings via bidirectional alignment of dimensions with semantic concepts

  2022cites this paper
• Revisiting Supervised Word Embeddings

  2022cites this paper
• Interpreting Deep Learning Models in Natural Language Processing: A Review

  2021cites this paper
• archer at SemEval-2021 Task 1: Contextualising Lexical Complexity

  2021cites this paper
• Spoken words as biomarkers: using machine learning to gain insight into communication as a predictor of anxiety

  2020cites this paper
• Conception: Multilingually-Enhanced, Human-Readable Concept Vector Representations

  2020cites this paper
• A Mixture-of-Experts Model for Learning Multi-Facet Entity Embeddings

  2020cites this paper
• Word Equations: Inherently Interpretable Sparse Word Embeddings through Sparse Coding

  2020cites this paper
• RPD: A Distance Function Between Word Embeddings

  2020cites this paper
• Discovering and Categorising Language Biases in Reddit

  2020cites this paper
• Inherently Interpretable Sparse Word Embeddings through Sparse Coding

  2020cites this paper
• CatE: Category-Name GuidedWord Embedding

  2019cites this paper
• Antonym-Synonym Classification Based on New Sub-space Embeddings

  2019influential citation
• Analyzing Structures in the Semantic Vector Space: A Framework for Decomposing Word Embeddings

  2019cites this paper
• Interpretable Word Embeddings via Informative Priors

  2019cites this paper
• Discriminative Topic Mining via Category-Name Guided Text Embedding

  2019cites this paper
• Exploiting Flickr meta-data for predicting environmental features

  2019cites this paper
• Predicting Concreteness and Imageability of Words Within and Across Languages via Word Embeddings

  2018cites this paper
• On Cognitive Preferences and the Interpretability of Rule-based Models

  2018cites this paper
• Embedding Geographic Locations for Modelling the Natural Environment using Flickr Tags and Structured Data

  2018cites this paper
• Modelling Salient Features as Directions in Fine-Tuned Semantic Spaces

  2018cites this paper
• Refining Pretrained Word Embeddings Using Layer-wise Relevance Propagation

  2018cites this paper
• Uncovering Divergent Linguistic Information in Word Embeddings with Lessons for Intrinsic and Extrinsic Evaluation

  2018cites this paper
• On cognitive preferences and the plausibility of rule-based models

  2018cites this paper
• Relation Induction in Word Embeddings Revisited

  2018cites this paper
• Learning Word Vectors with Linear Constraints: A Matrix Factorization Approach

  2018cites this paper
• Word embeddings quantify 100 years of gender and ethnic stereotypes

  2017cites this paper
• Distributed representations for fine-grained entity typing

  2017cites this paper
• Discovering Stylistic Variations in Distributional Vector Space Models via Lexical Paraphrases

  2017cites this paper
• Parameter Free Hierarchical Graph-Based Clustering for Analyzing Continuous Word Embeddings

  2017cites this paper
• Modeling Context Words as Regions: An Ordinal Regression Approach to Word Embedding

  2017cites this paper
• Interactive Data Analytics for the Humanities

  2017cites this paper
• Rotations and Interpretability of Word Embeddings: The Case of the Russian Language

  2017cites this paper
• Sentence matrix normalization using most likely n-grams vector

  2017cites this paper
• How to evaluate word embeddings? On importance of data efficiency and simple supervised tasks

  2017cites this paper
• Probabilistic Relation Induction in Vector Space Embeddings

  2017cites this paper
• Graph-Based Bilingual Word Embedding for Statistical Machine Translation

  2016cites this paper
• Exploring Distributional Representations and Machine Translation for Aspect-based Cross-lingual Sentiment Classification.

  2016cites this paper
• Intrinsic Subspace Evaluation of Word Embedding Representations

  2016cites this paper