Evaluation of Word Vector Representations by Subspace Alignment

Unsupervisedly learned word vectors have proven to provide exceptionally effective features in many NLP tasks. Most common intrinsic evaluations of vector quality measure correlation with similarity judgments. However, these often correlate poorly with how well the learned representations perform as features in downstream evaluation tasks. We present QVEC—a computationally inexpensive intrinsic evaluation measure of the quality of word embeddings based on alignment to a matrix of features extracted from manually crafted lexical resources—that obtains strong correlation with performance of the vectors in a battery of downstream semantic evaluation tasks.1

• Publication year

  2015

• Venue

  Conference on Empirical Methods in Natural Language Processing

• Publication date

  2015-09-01

• Fields of study

  Computer Science

• Identifiers
  DOI 10.18653/v1/D15-1243
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Evaluation methods for unsupervised word embeddings

  2015cited by this paper
• Not All Contexts Are Created Equal: Better Word Representations with Variable Attention

  2015cited by this paper
• Two/Too Simple Adaptations of Word2Vec for Syntax Problems

  2015cited by this paper
• Non-distributional Word Vector Representations

  2015cited by this paper
• Linguistic Structured Sparsity in Text Categorization

  2014cited by this paper
• Metaphor Detection with Cross-Lingual Model Transfer

  2014cited by this paper
• GloVe: Global Vectors for Word Representation

  2014cited by this paper
• SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation

  2014cited by this paper
• Community Evaluation and Exchange of Word Vectors at wordvectors.org

  2014cited by this paper
• Revisiting Embedding Features for Simple Semi-supervised Learning

  2014cited by this paper
• Tailoring Continuous Word Representations for Dependency Parsing

  2014cited by this paper
• Augmenting English Adjective Senses with Supersenses

  2014cited by this paper
• PPDB: The Paraphrase Database

  2013influential reference
• Fish Transporters and Miracle Homes: How Compositional Distributional Semantics can Help NP Parsing

  2013cited by this paper
• Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

  2013influential reference
• Supersense Tagging for Arabic: the MT-in-the-Middle Attack

  2013cited by this paper
• Efficient Estimation of Word Representations in Vector Space

  2013cited by this paper
• Distributional Semantics in Technicolor

  2012cited by this paper
• Word Representations: A Simple and General Method for Semi-Supervised Learning

  2010cited by this paper
• Unsupervised All-words Word Sense Disambiguation with Grammatical Dependencies

  2008cited by this paper
• Broad-Coverage Sense Disambiguation and Information Extraction with a Supersense Sequence Tagger

  2006cited by this paper
• Placing search in context: the concept revisited

  2002cited by this paper
• Book Reviews: WordNet: An Electronic Lexical Database

  1999cited by this paper
• A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge.

  1997cited by this paper
• A Semantic Concordance

  1993cited by this paper
• Mathematics without Numbers

  1993cited by this paper
• Word Association Norms, Mutual Information, and Lexicography

  1989cited by this paper

• Two statistical problems for multivariate mixture distributions

  2025cites this paper
• By Tying Embeddings You Are Assuming the Distributional Hypothesis

  2024cites this paper
• A Comprehensive Analysis of Static Word Embeddings for Turkish

  2024cites this paper
• Linguistically Informed Language Generation: A Multi-faceted Approach

  2024cites this paper
• Encoding and Decoding Graph Representations of Natural Language

  2024cites this paper
• Rethinking Node Representation Interpretation through Relation Coherence

  2024cites this paper
• ANALOGICAL - A Novel Benchmark for Long Text Analogy Evaluation in Large Language Models

  2023cites this paper
• ANALOGICAL - A New Benchmark for Analogy of Long Text for Large Language Models

  2023cites this paper
• What Do Self-Supervised Speech Models Know About Words?

  2023cites this paper
• A Neighbourhood-Aware Differential Privacy Mechanism for Static Word Embeddings

  2023cites this paper
• Distance geometry for word representations and applications

  2023cites this paper
• Exploring Pandemics Events on Twitter by Using Sentiment Analysis and Topic Modelling

  2022cites this paper
• Multi-objective search for gender-fair and semantically correct word embeddings

  2022cites this paper
• Information Theory–based Compositional Distributional Semantics

  2022cites this paper
• Domain-Specific Word Embeddings with Structure Prediction

  2022cites this paper
• Distributional Word Vectors as Semantic Maps Framework

  2022cites this paper
• Text Tonality Classification Using a Hybrid Convolutional Neural Network with Parallel and Sequential Connections Between Layers

  2022cites this paper
• Neural Representation Learning for Scribal Hands of Linear B

  2021influential citation
• Changing the Geometry of Representations: α-Embeddings for NLP Tasks

  2021cites this paper
• Fine-Tuning Word Embeddings for Hierarchical Representation of Data Using a Corpus and a Knowledge Base for Various Machine Learning Applications

  2021cites this paper
• MEET-LM: A method for embeddings evaluation for taxonomic data in the labour market

  2021cites this paper
• PASTRIE: A Corpus of Prepositions Annotated with Supersense Tags in Reddit International English

  2021cites this paper
• SBERT-WK: A Sentence Embedding Method by Dissecting BERT-Based Word Models

  2020cites this paper
• Taxonomizer: Interactive Construction of Fully Labeled Hierarchical Groupings from Attributes of Multivariate Data

  2020cites this paper
• Understanding the Semantic Content of Sparse Word Embeddings Using a Commonsense Knowledge Base

  2020cites this paper
• Similarity Judgment Within and Across Categories: A Comprehensive Model Comparison

  2020cites this paper
• Intrinsic Evaluation of Word Embeddings for Turkish

  2020cites this paper
• Word embeddings for biomedical natural language processing: A survey

  2020cites this paper
• A correlation-based entity embedding approach for robust entity linking

  2020cites this paper
• Performance Evaluation of Word Representation Techniques using Deep Learning Methods

  2020cites this paper
• Explaining Word Embeddings via Disentangled Representation

  2020cites this paper
• Low-Resource Adaptation of Neural NLP Models

  2020cites this paper
• Joint approaches for learning word representations from text corpora and knowledge bases

  2020cites this paper
• Distance geometry for word embeddings

  2020cites this paper
• Measuring the Semantic Stability of Word Embedding

  2020cites this paper
• CogniFNN: A Fuzzy Neural Network Framework for Cognitive Word Embedding Evaluation

  2020cites this paper
• Discrete Word Embedding for Logical Natural Language Understanding

  2020cites this paper
• Evaluating Natural Alpha Embeddings on Intrinsic and Extrinsic Tasks

  2020cites this paper
• Why is penguin more similar to polar bear than to sea gull? Analyzing conceptual knowledge in distributional models

  2020cites this paper
• Towards a Gold Standard for Evaluating Danish Word Embeddings

  2020cites this paper
• On the Correlation of Word Embedding Evaluation Metrics

  2020cites this paper
• Optimising word embeddings with search-based approaches

  2020cites this paper
• Are All Good Word Vector Spaces Isomorphic?

  2020cites this paper
• Multilingual Probing Tasks for Word Representations

  2020cites this paper
• From static to dynamic word representations: a survey

  2020cites this paper
• Massively Multilingual Sparse Word Representations

  2020cites this paper
• Vector-Space Models of Semantic Representation From a Cognitive Perspective: A Discussion of Common Misconceptions

  2019cites this paper
• Evaluating word embedding models: methods and experimental results

  2019cites this paper
• On the Idiosyncrasies of the Mandarin Chinese Classifier System

  2019influential citation
• Arabic Poem Generation with Hierarchical Recurrent Attentional Network

  2019cites this paper
• Linguistic Knowledge and Transferability of Contextual Representations

  2019cites this paper
• LINSPECTOR: Multilingual Probing Tasks for Word Representations

  2019cites this paper
• Gating Mechanisms for Combining Character and Word-level Word Representations: an Empirical Study

  2019cites this paper
• Word Embeddings and Gender Stereotypes in Swedish and English

  2019cites this paper
• Relational Word Embeddings

  2019cites this paper
• A Resource-Free Evaluation Metric for Cross-Lingual Word Embeddings Based on Graph Modularity

  2019cites this paper
• Does BLEU Score Work for Code Migration?

  2019cites this paper
• Intrinsic Evaluation for English–Tamil Bilingual Word Embeddings

  2019cites this paper
• Author Profiling using Semantic and Syntactic Features

  2019cites this paper
• Getting in Shape: Word Embedding SubSpaces

  2019cites this paper
• dish2vec: A Comparison of Word Embedding Methods in an Unsupervised Setting

  2019cites this paper
• Word embedding, neural networks and text classification: what is the state-of-the-art?

  2019cites this paper
• Dynamic Task-Specific Factors for Meta-Embedding

  2019cites this paper
• Exploiting Cross-Lingual Representations For Natural Language Processing

  2019cites this paper
• Hypergraph Partitioning With Embeddings

  2019cites this paper
• An evaluation of Czech word embeddings

  2019cites this paper
• Improving Word Representations: A Sub-sampled Unigram Distribution for Negative Sampling

  2019cites this paper
• A survey of word embeddings for clinical text

  2019cites this paper
• Interpretability of Hungarian embedding spaces using a knowledge base

  2019cites this paper
• Explorations in Word Embeddings : graph-based word embedding learning and cross-lingual contextual word embedding learning. (Explorations de plongements lexicaux : apprentissage de plongements à base de graphes et apprentissage de plongements contextuels multilingues)

  2019influential citation
• Evaluating the Consistency of Word Embeddings from Small Data

  2019cites this paper
• Détection de signaux faibles dans des masses de données faiblement structurées

  2019cites this paper
• Towards interpretable, data-derived distributional meaning representations for reasoning: A dataset of properties and concepts

  2019cites this paper
• Context-Aware Credit Card Fraud Detection

  2019cites this paper
• NAACL HLT 2018 Subword and Character LEvel Models in NLP Proceedings of the Second Workshop

  2018cites this paper
• Frame Semantics across Languages: Towards a Multilingual FrameNet

  2018cites this paper
• Firearms and Tigers are Dangerous, Kitchen Knives and Zebras are Not: Testing whether Word Embeddings Can Tell

  2018cites this paper
• Dynamic Meta-Embeddings for Improved Sentence Representations

  2018cites this paper
• Improving Vector Space Word Representations Via Kernel Canonical Correlation Analysis

  2018cites this paper
• Bridging CNNs, RNNs, and Weighted Finite-State Machines

  2018cites this paper
• Context-Attentive Embeddings for Improved Sentence Representations

  2018cites this paper
• A Survey of Word Embeddings Evaluation Methods

  2018influential citation
• Research and Analysis of Browsing Habits of New Media Audiences on the Chinese Campus

  2018cites this paper
• Delta Embedding Learning

  2018influential citation
• A Universal Semantic Space

  2018cites this paper
• QUESTION PREDICTION BASED ON ITS CONTEXT, WORD EMBEDDINGS AND NEURAL NETWORKS

  2018influential citation
• Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning

  2018cites this paper
• A Stronger Baseline for Multilingual Word Embeddings

  2018cites this paper
• Analysis of Short Text Classification strategies using Out of-domain Vocabularies

  2018cites this paper
• From Word to Sense Embeddings: A Survey on Vector Representations of Meaning

  2018cites this paper
• Estudio de word embeddings y métodos de generación de meta embeddings

  2018cites this paper
• Multilingual Embeddings Jointly Induced from Contexts and Concepts: Simple, Strong and Scalable

  2018cites this paper
• Multi-lingual Common Semantic Space Construction via Cluster-consistent Word Embedding

  2018influential citation
• Incorporating Subword Information into Matrix Factorization Word Embeddings

  2018cites this paper
• Bio-SimVerb and Bio-SimLex: wide-coverage evaluation sets of word similarity in biomedicine

  2018cites this paper
• What’s in Your Embedding, And How It Predicts Task Performance

  2018cites this paper
• Neural Random Projections for Language Modelling

  2018cites this paper
• Batch IS NOT Heavy: Learning Word Representations From All Samples

  2018influential citation
• Incorporating Latent Meanings of Morphological Compositions to Enhance Word Embeddings

  2018cites this paper
• Investigating Inner Properties of Multimodal Representation and Semantic Compositionality with Brain-based Componential Semantics

  2017cites this paper