Unsupervised Learning of Distributional Relation Vectors

Word embedding models such as GloVe rely on co-occurrence statistics to learn vector representations of word meaning. While we may similarly expect that co-occurrence statistics can be used to capture rich information about the relationships between different words, existing approaches for modeling such relationships are based on manipulating pre-trained word vectors. In this paper, we introduce a novel method which directly learns relation vectors from co-occurrence statistics. To this end, we first introduce a variant of GloVe, in which there is an explicit connection between word vectors and PMI weighted co-occurrence vectors. We then show how relation vectors can be naturally embedded into the resulting vector space.

• Publication year

  2018

• Venue

  Annual Meeting of the Association for Computational Linguistics

• Publication date

  2018-07-01

• Fields of study

  Computer Science

• Identifiers
  DOI 10.18653/v1/P18-1003
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Learning Distributed Representations of Sentences from Unlabelled Data

  2016influential reference
• Neural Relation Extraction with Selective Attention over Instances

  2016cited by this paper
• Word Embeddings, Analogies, and Machine Learning: Beyond king - man + woman = queen

  2016cited by this paper
• D-GloVe: A Feasible Least Squares Model for Estimating Word Embedding Densities

  2016influential reference
• Siamese CBOW: Optimizing Word Embeddings for Sentence Representations

  2016cited by this paper
• Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Paths

  2015cited by this paper
• Take and Took, Gaggle and Goose, Book and Read: Evaluating the Utility of Vector Differences for Lexical Relation Learning

  2015influential reference
• Task-Oriented Learning of Word Embeddings for Semantic Relation Classification

  2015cited by this paper
• Jointly Embedding Relations and Mentions for Knowledge Population

  2015cited by this paper
• Classifying Relations by Ranking with Convolutional Neural Networks

  2015cited by this paper
• Knowledge Graph Embedding by Translating on Hyperplanes

  2014influential reference
• Relation Classification via Convolutional Deep Neural Network

  2014cited by this paper
• GloVe: Global Vectors for Word Representation

  2014influential reference
• Knowledge Graph and Text Jointly Embedding

  2014influential reference
• Distributed Representations of Sentences and Documents

  2014cited by this paper
• Connecting Language and Knowledge Bases with Embedding Models for Relation Extraction

  2013cited by this paper
• Combining Heterogeneous Models for Measuring Relational Similarity

  2013cited by this paper
• Efficient Estimation of Word Representations in Vector Space

  2013influential reference
• Distributed Representations of Words and Phrases and their Compositionality

  2013influential reference
• Entailment above the word level in distributional semantics

  2012cited by this paper
• Semantic Compositionality through Recursive Matrix-Vector Spaces

  2012cited by this paper
• SemEval-2012 Task 2: Measuring Degrees of Relational Similarity

  2012cited by this paper
• UTD: Determining Relational Similarity Using Lexical Patterns

  2012cited by this paper
• Two Multivariate Generalizations of Pointwise Mutual Information

  2011cited by this paper
• Learning Structured Embeddings of Knowledge Bases

  2011cited by this paper
• Composition in Distributional Models of Semantics

  2010cited by this paper
• Toward an Architecture for Never-Ending Language Learning

  2010cited by this paper
• Modeling Relations and Their Mentions without Labeled Text

  2010cited by this paper
• Distant supervision for relation extraction without labeled data

  2009cited by this paper
• SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals

  2009cited by this paper
• Open Information Extraction from the Web

  2007cited by this paper
• Preemptive Information Extraction using Unrestricted Relation Discovery

  2006cited by this paper
• The Automatic Content Extraction (ACE) Program – Tasks, Data, and Evaluation

  2004cited by this paper
• Handbook of Natural Language Processing

  2001cited by this paper
• Snowball: extracting relations from large plain-text collections

  2000cited by this paper
• Extracting Patterns and Relations from the World Wide Web

  1998cited by this paper

• Large Language Models for Creation, Enrichment and Evaluation of Taxonomic Graphs

  2026cites this paper
• TaxoLLaMA: WordNet-based Model for Solving Multiple Lexical Sematic Tasks

  2024cites this paper
• RelBERT: Embedding Relations with Language Models

  2023cites this paper
• Predicting sentiment analysis for Web users with a deep learning approach

  2023cites this paper
• Consumer sentiment analysis with aspect fusion and GAN‐BERT aided adversarial learning

  2023cites this paper
• Distilling Relation Embeddings from Pretrained Language Models

  2021cites this paper
• Classifying and completing word analogies by machine learning

  2021cites this paper
• A Critical Review of Network-Based and Distributional Approaches to Semantic Memory Structure and Processes

  2021cites this paper
• An Improved Model for Analyzing Textual Sentiment Based on a Deep Neural Network Using Multi-Head Attention Mechanism

  2021cites this paper
• Lexical Relation Mining in Neural Word Embeddings

  2020cites this paper
• Understanding the Source of Semantic Regularities in Word Embeddings

  2020cites this paper
• Représentation vectorielle de paires de verbes pour la prédiction de relations lexicales (Verb-pairs embeddings for discourse relation prediction)

  2020cites this paper
• Short‐text feature expansion and classification based on nonnegative matrix factorization

  2020cites this paper
• Context-Guided Self-supervised Relation Embeddings

  2019cites this paper
• Attention-based long short-term memory network using sentiment lexicon embedding for aspect-level sentiment analysis in Korean

  2019cites this paper
• Distributional Effects of Gender Contrasts Across Categories

  2019cites this paper
• Relational Word Embeddings

  2019cites this paper
• SphereRE: Distinguishing Lexical Relations with Hyperspherical Relation Embeddings

  2019cites this paper
• Collocation Classification with Unsupervised Relation Vectors

  2019cites this paper
• Relation Module for Non-answerable Prediction on Question Answering

  2019cites this paper
• Relation Module for Non-Answerable Predictions on Reading Comprehension

  2019cites this paper
• Inducing Relational Knowledge from BERT

  2019cites this paper
• From Word to Sense Embeddings: A Survey on Vector Representations of Meaning

  2018cites this paper
• Generalizing Representations of Lexical Semantic Relations

  2018influential citation
• pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference

  2018cites this paper
• Neural Latent Relational Analysis to Capture Lexical Semantic Relations in a Vector Space

  2018cites this paper