Improving Language Understanding by Generative Pre-Training

No abstract is available for this paper.

• Publication year

  2018

• Venue

  Unknown venue

• Publication date

  Unknown publication date

• Fields of study

  Linguistics, Computer Science

• Identifiers

  No identifiers available.

• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning

  2018cited by this paper
• Constituency Parsing with a Self-Attentive Encoder

  2018cited by this paper
• SciTaiL: A Textual Entailment Dataset from Science Question Answering

  2018cited by this paper
• Stochastic Answer Networks for Natural Language Inference

  2018cited by this paper
• Multi-range Reasoning for Machine Comprehension

  2018cited by this paper
• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

  2018influential reference
• An efficient framework for learning sentence representations

  2018cited by this paper
• Generating Wikipedia by Summarizing Long Sequences

  2018cited by this paper
• Deep Contextualized Word Representations

  2018influential reference
• A Simple and Effective Approach to the Story Cloze Test

  2018cited by this paper
• Universal Language Model Fine-tuning for Text Classification

  2018cited by this paper
• When and Why Are Pre-Trained Word Embeddings Useful for Neural Machine Translation?

  2018cited by this paper
• Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

  2017cited by this paper
• Semi-supervised Multitask Learning for Sequence Labeling

  2017cited by this paper
• Towards Human-level Machine Reading Comprehension: Reasoning and Inference with Multiple Strategies

  2017cited by this paper
• A Simple but Tough-to-Beat Baseline for Sentence Embeddings

  2017cited by this paper
• GPU Kernels for Block-Sparse Weights

  2017cited by this paper
• Quora Question Pairs

  2017cited by this paper
• ECNU at SemEval-2017 Task 1: Leverage Kernel-based Traditional NLP features and Neural Networks to Build a Universal Model for Multilingual and Cross-lingual Semantic Textual Similarity

  2017cited by this paper
• Fixing Weight Decay Regularization in Adam

  2017cited by this paper
• A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference

  2017influential reference
• Unsupervised Machine Translation Using Monolingual Corpora Only

  2017cited by this paper
• SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation

  2017cited by this paper
• Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning

  2017cited by this paper
• RACE: Large-scale ReAding Comprehension Dataset From Examinations

  2017influential reference
• Semi-supervised sequence tagging with bidirectional language models

  2017cited by this paper
• Learned in Translation: Contextualized Word Vectors

  2017cited by this paper
• Story Comprehension for Predicting What Happens Next

  2017cited by this paper
• LSDSem 2017 Shared Task: The Story Cloze Test

  2017influential reference
• Attention is All you Need

  2017influential reference
• A Compare-Propagate Architecture with Alignment Factorization for Natural Language Inference

  2017cited by this paper
• Layer Normalization

  2016cited by this paper
• Unsupervised Pretraining for Sequence to Sequence Learning

  2016cited by this paper
• Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction

  2016cited by this paper
• SQuAD: 100,000+ Questions for Machine Comprehension of Text

  2016cited by this paper
• Bridging Nonlinearities and Stochastic Regularizers with Gaussian Error Linear Units

  2016cited by this paper
• Neural Machine Translation of Rare Words with Subword Units

  2015cited by this paper
• Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books

  2015cited by this paper
• Skip-Thought Vectors

  2015cited by this paper
• Semi-supervised Sequence Learning

  2015cited by this paper
• Teaching Machines to Read and Comprehend

  2015cited by this paper
• A large annotated corpus for learning natural language inference

  2015cited by this paper
• Reasoning about Entailment with Neural Attention

  2015cited by this paper
• Convolutional Neural Networks for Sentence Classification

  2014cited by this paper
• Distributed Representations of Sentences and Documents

  2014cited by this paper
• GloVe: Global Vectors for Word Representation

  2014cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• A Fast and Accurate Dependency Parser using Neural Networks

  2014cited by this paper
• Learning Entity Representation for Entity Disambiguation

  2013cited by this paper
• Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

  2013cited by this paper
• Discriminative Improvements to Distributional Sentence Similarity

  2013cited by this paper
• Distributed Representations of Words and Phrases and their Compositionality

  2013cited by this paper
• Resolving Complex Cases of Definite Pronouns: The Winograd Schema Challenge

  2012cited by this paper
• Natural Language Processing (Almost) from Scratch

  2011cited by this paper
• Why Does Unsupervised Pre-training Help Deep Learning?

  2010cited by this paper
• Roles of Pre-Training and Fine-Tuning in Context-Dependent DBN-HMMs for Real-World Speech Recognition

  2010cited by this paper
• The Sixth PASCAL Recognizing Textual Entailment Challenge

  2009influential reference
• A unified architecture for natural language processing: deep neural networks with multitask learning

  2008cited by this paper
• The Fourth PASCAL Recognizing Textual Entailment Challenge

  2008cited by this paper
• Extracting and composing robust features with denoising autoencoders

  2008cited by this paper
• Semi-Supervised Sequential Labeling and Segmentation Using Giga-Word Scale Unlabeled Data

  2008cited by this paper
• --1 CONTENTS

  2006cited by this paper
• Greedy Layer-Wise Training of Deep Networks

  2006cited by this paper
• Semi-Supervised Conditional Random Fields for Improved Sequence Segmentation and Labeling

  2006cited by this paper
• A Fast Learning Algorithm for Deep Belief Nets

  2006cited by this paper
• Efficient Learning of Sparse Representations with an Energy-Based Model

  2006cited by this paper
• Semi-Supervised Text Classification Using EM

  2006cited by this paper
• Influence of cultivation temperature on the ligninolytic activity of selected fungal strains

  2006cited by this paper
• Automatically Constructing a Corpus of Sentential Paraphrases

  2005cited by this paper
• Semi-Supervised Learning for Natural Language

  2005cited by this paper
• A Stochastic Approximation Method

  1951cited by this paper

• A systematic review of molecular representation learning foundation models

  2026cites this paper
• NEuRT: A Transformer-Based Model for Explainable Neuronal Activity Analysis

  2026cites this paper
• Optimizing Mixture-of-Experts Inference Time via Model Deployment and Communication Scheduling

  2026cites this paper
• Self-Supervised Learning for Extracting Interpersonal Dynamics in Video Robbery Detection

  2026cites this paper
• Layer-Parallel Training for Transformers

  2026cites this paper
• A systematic evaluation and benchmarking of text summarization methods for biomedical literature: From word-frequency methods to language models

  2026cites this paper
• Generative AI-Driven Metaverse: The Promises and Challenges of AI-Generated Content

  2026cites this paper
• MobiFM: A Foundation Model for Mobile Data Forecasting

  2026cites this paper
• Language models for environmental, social, and governance analysis: A review

  2026cites this paper
• An Introduction to Pathology Foundation Models

  2026influential citation
• GPT-4o and OpenAI o1 Performance on the 2024 Spanish Competitive Medical Specialty Access Examination: Cross-Sectional Quantitative Evaluation Study

  2026cites this paper
• Generative AI for Learning and Teaching Arabic: Opportunities, Challenges, and Ethical Considerations

  2026cites this paper
• Developing an Interpretation Argument for AI-Enhanced Scenario-Based Language Assessments Through a Learning-Oriented Language Assessment Framework

  2026cites this paper
• MCQs Generation With Large Language Models: A Survey of Methodologies, Evolution, and Open Research Issues

  2026influential citation
• Unsupervised Action Segmentation via Multi-Scale Temporal-Interaction Enhancement

  2026cites this paper
• Decision-Making Large Language Model for Wireless Communication: A Comprehensive Survey on Key Techniques

  2026cites this paper
• Text content analysis techniques that marketing managers must know: a comprehensive literature review and practical applications

  2026cites this paper
• TIDES: Traffic Intelligence With DeepSeek-Enhanced Spatial–Temporal Prediction

  2026cites this paper
• Antidote or Placebo? Unraveling the Efficacy of Neuron Coverage Criteria on Testing Transformer-based Language Models

  2026cites this paper
• VAR RL Done Right: Tackling Asynchronous Policy Conflicts in Visual Autoregressive Generation

  2026cites this paper
• ChatGPT-4o as an automated scoring tool for writing assessment: Strengths and weaknesses

  2026cites this paper
• PRISM: A Unified Framework for Post-Training LLMs Without Verifiable Rewards

  2026cites this paper
• CSR-RAG: An Efficient Retrieval System for Text-to-SQL on the Enterprise Scale

  2026cites this paper
• Can a Unimodal Language Agent Provide Preferences to Tune a Multimodal Vision-Language Model?

  2026cites this paper
• Do You Understand How I Feel?: Towards Verified Empathy in Therapy Chatbots

  2026cites this paper
• FiLoRA: Parameter-Efficient Fine-Tuning With Fisher Information-Guided Low-Rank Adaptation

  2026cites this paper
• Multilingual-To-Multimodal (M2M): Unlocking New Languages with Monolingual Text

  2026cites this paper
• CYBY24 and Step-Wise Model for Thread-Based Fine-Grained Cyberbullying Detection

  2026cites this paper
• Towards Autonomous PHM: An Application to Turboshaft Engine Torque Prediction

  2026cites this paper
• Generalized ECG Heartbeat Classification Using Time-Series Transformers

  2026cites this paper
• From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications

  2026cites this paper
• RoboCleaner: Robotic Tabletop Cleaning via VLM-Powered Multi-Agent Collaboration

  2026cites this paper
• Multi-Domain Dialogue State Tracking With Large Language Model Rationale and Disentangled Domain-Slot Attention

  2026cites this paper
• Regression in Earth Observation: Are vision–language models up to the challenge?

  2026cites this paper
• SemanticLog: Towards Effective and Efficient Large-Scale Semantic Log Parsing

  2026cites this paper
• Efficient KV Cache Spillover Management on Memory-Constrained GPU for LLM Inference

  2026cites this paper
• Multimodal large models for smart agriculture: Frameworks and applications

  2026cites this paper
• Distformer: Time–Frequency Feature Distribution-Aware Transformer for Long-Term Time Series Prediction

  2026cites this paper
• Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Survey

  2026cites this paper
• Online Finetuning Decision Transformers with Pure RL Gradients

  2026cites this paper
• HyperJoin: LLM-augmented Hypergraph Link Prediction for Joinable Table Discovery

  2026cites this paper
• Output Embedding Centering for Stable LLM Pretraining

  2026cites this paper
• The performances of the Chinese and U.S. Large Language Models on the Topic of Chinese Culture

  2026cites this paper
• Window-based Membership Inference Attacks Against Fine-tuned Large Language Models

  2026cites this paper
• FLEx: Language Modeling with Few-shot Language Explanations

  2026influential citation
• CRUNet-MR-Univ: A Foundation Model for Diverse Cardiac MRI Reconstruction

  2026cites this paper
• CyberGFM: Graph Foundation Models for Lateral Movement Detection in Enterprise Networks

  2026cites this paper
• A Brief Review of Artificial Intelligence in Living Kidney Donation

  2026cites this paper
• Artificial Entanglement in the Fine-Tuning of Large Language Models

  2026influential citation
• Engineering of Hallucination in Generative AI: It's not a Bug, it's a Feature

  2026cites this paper
• Markovian Pre-Trained Transformer for Next-Item Recommendation

  2026cites this paper
• Silence the Judge: Reinforcement Learning with Self-Verifier via Latent Geometric Clustering

  2026cites this paper
• An Axiomatic Approach to General Intelligence: SANC(E3) - Self-organizing Active Network of Concepts with Energy E3

  2026cites this paper
• Universal computation is intrinsic to language model decoding

  2026cites this paper
• Linear Complexity Self-Supervised Learning for Music Understanding with Random Quantizer

  2026cites this paper
• A comprehensive survey of genome language models in bioinformatics

  2026cites this paper
• Single-Stage Huffman Encoder for ML Compression

  2026cites this paper
• Unlabeled Data Can Provably Enhance In-Context Learning of Transformers

  2026cites this paper
• Trustworthy AI-IoT for Citizen-Centric Smart Cities: The IMTPS Framework for Intelligent Multimodal Crowd Sensing

  2026influential citation
• Enhancing large language models for knowledge graph question answering via multi-granularity knowledge injection and structured reasoning path-augmented prompting

  2026cites this paper
• Advancing cyberbullying detection in low-resource languages: a transformer- stacking framework for Bengali

  2026cites this paper
• Compressing What Matters: Neuron Importance Meets Data-Aware Low Rank Approximation for Language Model Compression

  2026cites this paper
• An Effective Masked Transformer Model for Automatic Modulation Recognition

  2026influential citation
• A Survey on Applications of Large Language Model-Driven Digital Twins for Intelligent Network Optimization

  2026cites this paper
• Missing Data Recovery Methods on Multivariate Time Series in IoT: A Comprehensive Survey

  2026cites this paper
• Federated Prompt-Based Decision Transformer for Resource Allocation of Customized VR Streaming in Mobile Edge Computing

  2026cites this paper
• MSAF: Multimodal Sentiment Detection via Multiscale Adaptive Fusion

  2026cites this paper
• Disaster Detection on the Fly: Optimized Transformers for UAVs

  2026cites this paper
• A 3.51 TOPS/mm2 Transformer Accelerator Exploiting Bipolar Sparsity and Approximate Gating

  2026cites this paper
• ComAI: The Convergence of Communication and Artificial Intelligence

  2026cites this paper
• Mujaz: A Summarization-Based Approach for Normalized Vulnerability Description

  2026cites this paper
• Evidence from Formal Logical Reasoning Reveals that the Language of Thought is not Natural Language

  2026cites this paper
• Disguise While Defense: Avoid Refusal Responses in LLM’s Defense via a Multi-Agent Attacker–Disguiser Game

  2026cites this paper
• Fine-Grained Scheduling of In-Network Aggregation Resources for Efficient Machine Learning Service

  2026cites this paper
• Continuous Matrix Transposition for a Subclass of Matrices Using Minimal Memory

  2026cites this paper
• Purified Zero-Shot Sketch-Based Image Retrieval

  2026cites this paper
• Reinforcement Learning with Backtracking Feedback

  2026cites this paper
• Offline reinforcement learning for badminton tactical decision-making

  2026cites this paper
• Distributed Modularization of Thought: Lets Small Rival Large LMs

  2026cites this paper
• Causal-ESC : Capture the Dynamics in Cause-and-Effect Detection for Emotional Support Conversation

  2026cites this paper
• Towards More Accurate App Review Response Generation with Fine-Grained Filtering

  2026cites this paper
• Artificial Intelligence in Academic Research Publishing: Updates, Controversies, and Considerations for Pregnancy and Perinatal Research.

  2026cites this paper
• Pilot Contamination Aware Transformer for Downlink Power Control in Cell-Free Massive MIMO Networks

  2026cites this paper
• Improving Flexible Image Tokenizers for Autoregressive Image Generation

  2026cites this paper
• IO-RAE: Information-Obfuscation Reversible Adversarial Example for Audio Privacy Protection

  2026cites this paper
• LIA: Supervised Fine-Tuning of Large Language Models for Automatic Issue Assignment

  2026cites this paper
• Language Models’ Hall of Mirrors Problem: Why AI Alignment Requires Peircean Semiosis

  2026influential citation
• Efficient Tuning Framework for Resource- Constrained Biomedical Question Answering

  2026cites this paper
• Reconstructing Item Characteristic Curves using Fine-Tuned Large Language Models

  2026cites this paper
• DIP: Dynamic In-Context Planner For Diffusion Language Models

  2026cites this paper
• Measuring accuracy of AI generated definitions using cosine similarity index across select GPT models

  2026cites this paper
• Prompt Tuning without Labeled Samples for Zero-Shot Node Classification in Text-Attributed Graphs

  2026cites this paper
• Enhancing Linguistic Competence of Language Models through Pre-training with Language Learning Tasks

  2026cites this paper
• Learning Latent Action World Models In The Wild

  2026cites this paper
• Open World Knowledge Aided Single-Cell Foundation Model with Robust Cross-Modal Cell-Language Pre-training

  2026cites this paper
• Dual-Phase LLM Reasoning: Self-Evolved Mathematical Frameworks

  2026influential citation
• Pantagruel: Unified Self-Supervised Encoders for French Text and Speech

  2026cites this paper
• Exploring generative artificial intelligence: a comprehensive guide

  2026cites this paper
• KALE: Enhancing Knowledge Manipulation in Large Language Models via Knowledge-aware Learning

  2026cites this paper
• KLSBench: Evaluating LLM Capabilities on Korean Literary Sinitic Texts in Historical Context

  2026cites this paper