Adversarial Generation of Natural Language

Generative Adversarial Networks (GANs) have gathered a lot of attention from the computer vision community, yielding impressive results for image generation. Advances in the adversarial generation of natural language from noise however are not commensurate with the progress made in generating images, and still lag far behind likelihood based methods. In this paper, we take a step towards generating natural language with a GAN objective alone. We introduce a simple baseline that addresses the discrete output space problem without relying on gradient estimators and show that it is able to achieve state-of-the-art results on a Chinese poem generation dataset. We present quantitative results on generating sentences from context-free and probabilistic context-free grammars, and qualitative language modeling results. A conditional version is also described that can generate sequences conditioned on sentence characteristics.

• Publication year

  2017

• Venue

  Rep4NLP@ACL

• Publication date

  2017-05-31

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.18653/v1/W17-2629arXiv 1705.10929
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• GENERATIVE ADVERSARIAL NETS

  2018influential reference
• Improved Training of Wasserstein GANs

  2017influential reference
• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks

  2017influential reference
• Toward Controlled Generation of Text

  2017cited by this paper
• Towards Principled Methods for Training Generative Adversarial Networks

  2017influential reference
• Controllable Text Generation

  2017cited by this paper
• Wasserstein GAN

  2017influential reference
• On the Quantitative Analysis of Decoder-Based Generative Models

  2016cited by this paper
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization

  2016cited by this paper
• Language Modeling with Gated Convolutional Networks

  2016cited by this paper
• Professor Forcing: A New Algorithm for Training Recurrent Networks

  2016cited by this paper
• Mode Regularized Generative Adversarial Networks

  2016cited by this paper
• Categorical Reparameterization with Gumbel-Softmax

  2016cited by this paper
• Sequence-to-Sequence Learning as Beam-Search Optimization

  2016cited by this paper
• SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient

  2016influential reference
• Least Squares Generative Adversarial Networks

  2016cited by this paper
• Adversarially Learned Inference

  2016cited by this paper
• Improved Techniques for Training GANs

  2016cited by this paper
• RECURRENT NEURAL NETWORKS

  2015cited by this paper
• Deep Residual Learning for Image Recognition

  2015cited by this paper
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

  2015cited by this paper
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks

  2015cited by this paper
• Conditional generative adversarial nets for convolutional face generation

  2015cited by this paper
• A Neural Attention Model for Abstractive Sentence Summarization

  2015cited by this paper
• Character-Aware Neural Language Models

  2015cited by this paper
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

  2015cited by this paper
• Character-level Convolutional Networks for Text Classification

  2015cited by this paper
• Conditional Generative Adversarial Nets

  2014cited by this paper
• Recurrent Neural Network Regularization

  2014cited by this paper
• Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

  2014cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014cited by this paper
• Chinese Poetry Generation with Recurrent Neural Networks

  2014cited by this paper
• Auto-Encoding Variational Bayes

  2013cited by this paper
• One billion word benchmark for measuring progress in statistical language modeling

  2013cited by this paper
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation

  2013cited by this paper
• Deep Sparse Rectifier Neural Networks

  2011cited by this paper
• Recurrent neural network based language model

  2010cited by this paper
• Rectified Linear Units Improve Restricted Boltzmann Machines

  2010cited by this paper
• Curriculum learning

  2009cited by this paper
• On the quantitative analysis of deep belief networks

  2008cited by this paper
• Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks

  2008cited by this paper
• Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

  2004cited by this paper
• ROUGE: A Package for Automatic Evaluation of Summaries

  2004cited by this paper
• A* Parsing: Fast Exact Viterbi Parse Selection

  2003cited by this paper
• Bleu: a Method for Automatic Evaluation of Machine Translation

  2002influential reference
• Long Short-Term Memory

  1997cited by this paper
• Automatic Grammar Induction and Parsing Free Text: A Transformation-Based Approach

  1993cited by this paper
• Building a Large Annotated Corpus of English: The Penn Treebank

  1993influential reference
• A Learning Algorithm for Continually Running Fully Recurrent Neural Networks

  1989cited by this paper
• An efficient context-free parsing algorithm

  1970cited by this paper

• Generative models for data-scarcity challenges in AlGaN DUV laser diodes: a first-of-its-kind study

  2026cites this paper
• Feedback Friction: LLMs Struggle to Fully Incorporate External Feedback

  2025cites this paper
• An ensemble-based transfer testing method for Large Language Models

  2025cites this paper
• GIT-CXR: End-to-End Transformer for Chest X-Ray Report Generation

  2025cites this paper
• GRADE: Replacing Policy Gradients with Backpropagation for LLM Alignment

  2025cites this paper
• LAKEGEN: A LLM-based Tabular Corpus Generator for Evaluating Dataset Discovery in Data Lakes

  2025cites this paper
• LeanAgent: Lifelong Learning for Formal Theorem Proving

  2024cites this paper
• Application of deep learning models to generate rich, dynamic and production-like test data

  2024cites this paper
• SELF-[IN]CORRECT: LLMs Struggle with Refining Self-Generated Responses

  2024cites this paper
• SumTra: A Differentiable Pipeline for Few-Shot Cross-Lingual Summarization

  2024cites this paper
• Perturbing and Backtracking Based Textual Adversarial Attack

  2024cites this paper
• GenerRNA: A generative pre-trained language model for de novo RNA design

  2024cites this paper
• SELF-[IN]CORRECT: LLMs Struggle with Discriminating Self-Generated Responses

  2024cites this paper
• A comprehensive survey on deep-learning-based visual captioning

  2023cites this paper
• RETRACTED: An enhanced affective computing-based framework using machine learning & medical IoT for the efficient pre-emptive decision-making of mental health problems

  2023cites this paper
• A Review of the Evaluation System for Curriculum Learning

  2023cites this paper
• Complexity Matters: Rethinking the Latent Space for Generative Modeling

  2023cites this paper
• Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models

  2023cites this paper
• GAN-LM: Generative Adversarial Network using Language Models for Downstream Applications

  2023cites this paper
• A Novel IoT Intrusion Detection Model Using 2dCNN-BiLSTM

  2023cites this paper
• Feature Extraction Using Deep Generative Models for Bangla Text Classification on a New Comprehensive Dataset

  2023cites this paper
• Evolutionary Curriculum Training for DRL-Based Navigation Systems

  2023cites this paper
• Bi-fidelity Variational Auto-encoder for Uncertainty Quantification

  2023cites this paper
• A big data approach towards sarcasm detection in Russian

  2023cites this paper
• OKAZ: A Deep-learning-Based System for Automatic Arabic Poem Generation

  2023influential citation
• Adversarial Conversational Shaping for Intelligent Agents

  2023cites this paper
• Contrastive Learning for Sign Language Recognition and Translation

  2023cites this paper
• Empowering NLG: Offline Reinforcement Learning for Informal Summarization in Online Domains

  2023cites this paper
• Generative Methods for Social Media Analysis

  2023cites this paper
• Generative Adversarial Networks (GANs) in Computer-Generated Imagery

  2023cites this paper
• Synthetic data generation for a longitudinal cohort study – evaluation, method extension and reproduction of published data analysis results

  2023cites this paper
• Utilizing Generative Adversarial Networks for Stable Structure Generation in Angry Birds

  2023cites this paper
• Comparison of deep convolution and least squares GANs for diabetic retinopathy image synthesis

  2023cites this paper
• Automatic Arabic Poem Generation with GPT-2

  2022cites this paper
• Generative Adversarial Networks for Creating Synthetic Nucleic Acid Sequences of Cat Genome

  2022influential citation
• Adversarial Machine Learning in Text Processing: A Literature Survey

  2022cites this paper
• Length-Based Curriculum Learning for Efficient Pre-training of Language Models

  2022cites this paper
• The Hilarious Bluffing GAN

  2022cites this paper
• Noun2Verb: Probabilistic Frame Semantics for Word Class Conversion

  2022cites this paper
• Are GANs overkill for NLP?

  2022cites this paper
• PoliSe: Reinforcing Politeness Using User Sentiment for Customer Care Response Generation

  2022cites this paper
• Language Models are Realistic Tabular Data Generators

  2022cites this paper
• Concatenate Word Embedding for Text to Image through Generative Adversarial Network

  2022cites this paper
• Learning to Perform Complex Tasks through Compositional Fine-Tuning of Language Models

  2022cites this paper
• Automatic Generation and Evaluation of Chinese Classical Poetry with Attention-Based Deep Neural Network

  2022cites this paper
• Text to Image Synthesis using Stacked Conditional Variational Autoencoders and Conditional Generative Adversarial Networks

  2022cites this paper
• Time-series Transformer Generative Adversarial Networks

  2022cites this paper
• CTTGAN: Traffic Data Synthesizing Scheme Based on Conditional GAN

  2022cites this paper
• Why GANs are overkill for NLP

  2022cites this paper
• Concealment Of Information In Images Using The Technique Of Steganography

  2021cites this paper
• Pre-training a BERT with Curriculum Learning by Increasing Block-Size of Input Text

  2021cites this paper
• GeneCGAN: A conditional generative adversarial network based on genetic tree for point cloud reconstruction

  2021cites this paper
• Sometimes We Want Ungrammatical Translations

  2021cites this paper
• When not to use an Adversarial Approach to Generative Modeling

  2021cites this paper
• On the Role of Corpus Ordering in Language Modeling

  2021cites this paper
• Deep Neural Networks and Tabular Data: A Survey

  2021cites this paper
• Fast simulation of a high granularity calorimeter by generative adversarial networks

  2021cites this paper
• Survey of Generative Methods for Social Media Analysis

  2021cites this paper
• PAR-GAN: Improving the Generalization of Generative Adversarial Networks Against Membership Inference Attacks

  2021cites this paper
• Improvised Theatre Alongside Artificial Intelligences

  2021cites this paper
• Sometimes We Want Translationese

  2021cites this paper
• Topic-to-Essay Generation with Comprehensive Knowledge Enhancement

  2021cites this paper
• Translating synthetic natural language to database queries with a polyglot deep learning framework

  2021cites this paper
• SPI-GAN: Towards Single-Pixel Imaging through Generative Adversarial Network

  2021cites this paper
• Curriculum Learning: A Survey

  2021cites this paper
• Adversarial training for few-shot text classification

  2021cites this paper
• Adversarial Machine Learning in Text Analysis and Generation

  2021cites this paper
• Missing value imputation in multivariate time series with end-to-end generative adversarial networks

  2021cites this paper
• GAN Computers Generate Arts? A Survey on Visual Arts, Music, and Literary Text Generation using Generative Adversarial Network

  2021cites this paper
• Auxiliary Classifier Generative Adversarial Network for Interictal Epileptiform Discharge Modeling and EEG Data Augmentation

  2021cites this paper
• KM4: Visual reasoning via Knowledge Embedding Memory Model with Mutual Modulation

  2021cites this paper
• Probabilistic Human-like Gesture Synthesis from Speech using GRU-based WGAN

  2021cites this paper
• Sentiment Analysis Method Based on Deep Learning in Adversarial Environment

  2021cites this paper
• Impossibility of Unambiguous Communication as a Source of Failure in AI Systems

  2021cites this paper
• High Energy Physics Calorimeter Detector Simulation Using Generative Adversarial Networks With Domain Related Constraints

  2021cites this paper
• Functorial Language Games for Question Answering

  2020cites this paper
• Deep Learning and Knowledge-Based Methods for Computer Aided Molecular Design - Toward a Unified Approach: State-of-the-Art and Future Directions

  2020cites this paper
• CausaLM: Causal Model Explanation Through Counterfactual Language Models

  2020cites this paper
• Distributional Discrepancy: A Metric for Unconditional Text Generation

  2020cites this paper
• Controlled Machine Text Generation of Football Articles

  2020cites this paper
• Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation

  2020cites this paper
• Wasserstein Generative Adversarial Networks Based Data Augmentation for Radar Data Analysis

  2020cites this paper
• Recurrent GANs Password Cracker For IoT Password Security Enhancement †

  2020cites this paper
• When Relation Networks meet GANs: Relation GANs with Triplet Loss

  2020cites this paper
• A Novel Framework for Selection of GANs for an Application

  2020cites this paper
• Point Encoder GAN: A deep learning model for 3D point cloud inpainting

  2020cites this paper
• Open-Ended Video Question Answering via Multi-Modal Conditional Adversarial Networks

  2020cites this paper
• Learning Generative Models using Transformations

  2020cites this paper
• Eleventh International Workshop Modelling and Reasoning in Context

  2020cites this paper
• Effect of Input Noise Dimension in GANs

  2020cites this paper
• Boosting Naturalness of Language in Task-oriented Dialogues via Adversarial Training

  2020cites this paper
• Single image super-resolution for whole slide image using convolutional neural networks and self-supervised color normalization

  2020cites this paper
• Use of Auxiliary Classifier Generative Adversarial Network in Touchstroke Authentication

  2020cites this paper
• Q UANTIFYING E XPOSURE B IAS FOR O PEN - ENDED L ANGUAGE G ENERATION

  2020cites this paper
• Newspaper article-based agent control in smart city simulations

  2020cites this paper
• A Preliminary Exploration of GANs for Keyphrase Generation

  2020cites this paper
• Expert-in-the-loop AI for Polymer Discovery

  2020cites this paper
• Curriculum Learning with Diversity for Supervised Computer Vision Tasks

  2020cites this paper
• Dual Attention-Based Adversarial Autoencoder for Attributed Network Embedding

  2020cites this paper
• Deep generative variational autoencoding for replay spoof detection in automatic speaker verification

  2020cites this paper