Image-to-Image Translation with Conditional Adversarial Networks

We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations. We demonstrate that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. Moreover, since the release of the pix2pix software associated with this paper, hundreds of twitter users have posted their own artistic experiments using our system. As a community, we no longer hand-engineer our mapping functions, and this work suggests we can achieve reasonable results without handengineering our loss functions either.

• Publication year

  2016

• Venue

  Computer Vision and Pattern Recognition

• Publication date

  2016-11-21

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1109/CVPR.2017.632arXiv 1611.07004
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Image Style Transfer using Convolutional Neural Networks

  2018cited by this paper
• GENERATIVE ADVERSARIAL NETS

  2018influential reference
• Generating Interpretable Images with Controllable Structure

  2017cited by this paper
• Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts

  2016cited by this paper
• Generative Visual Manipulation on the Natural Image Manifold

  2016cited by this paper
• Pixel-Level Domain Transfer

  2016cited by this paper
• Learning What and Where to Draw

  2016cited by this paper
• Generative Image Modeling Using Style and Structure Adversarial Networks

  2016cited by this paper
• Generative Adversarial Text to Image Synthesis

  2016cited by this paper
• Improved Techniques for Training GANs

  2016influential reference
• Context Encoders: Feature Learning by Inpainting

  2016cited by this paper
• Instance Normalization: The Missing Ingredient for Fast Stylization

  2016cited by this paper
• Energy-based Generative Adversarial Network

  2016cited by this paper
• Learning Temporal Transformations from Time-Lapse Videos

  2016cited by this paper
• Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

  2016cited by this paper
• Image Style Transfer Using Convolutional Neural Networks

  2016cited by this paper
• The Cityscapes Dataset for Semantic Urban Scene Understanding

  2016cited by this paper
• Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks

  2016cited by this paper
• Colorful Image Colorization

  2016influential reference
• Learning Representations for Automatic Colorization

  2016cited by this paper
• Perceptual Losses for Real-Time Style Transfer and Super-Resolution

  2016cited by this paper
• Generating Images with Perceptual Similarity Metrics based on Deep Networks

  2016cited by this paper
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

  2015cited by this paper
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

  2015cited by this paper
• Texture Synthesis Using Convolutional Neural Networks

  2015influential reference
• Holistically-Nested Edge Detection

  2015cited by this paper
• Visually Indicated Sounds

  2015cited by this paper
• Top-Down Learning for Structured Labeling with Convolutional Pseudoprior

  2015cited by this paper
• Autoencoding beyond pixels using a learned similarity metric

  2015cited by this paper
• Deep multi-scale video prediction beyond mean square error

  2015cited by this paper
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks

  2015cited by this paper
• Texture synthesis and the controlled generation of natural stimuli using convolutional neural networks

  2015cited by this paper
• Multispectral pedestrian detection: Benchmark dataset and baseline

  2015cited by this paper
• Conditional generative adversarial nets for convolutional face generation

  2015cited by this paper
• U-Net: Convolutional Networks for Biomedical Image Segmentation

  2015cited by this paper
• Transient attributes for high-level understanding and editing of outdoor scenes

  2014cited by this paper
• Fully convolutional networks for semantic segmentation

  2014cited by this paper
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs

  2014cited by this paper
• ImageNet Large Scale Visual Recognition Challenge

  2014cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• Fine-Grained Visual Comparisons with Local Learning

  2014cited by this paper
• Conditional Generative Adversarial Nets

  2014cited by this paper
• Learning to generate chairs with convolutional neural networks

  2014cited by this paper
• Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture

  2014cited by this paper
• Data-driven hallucination of different times of day from a single outdoor photo

  2013cited by this paper
• Spatial Pattern Templates for Recognition of Objects with Regular Structure

  2013cited by this paper
• Random Search for Hyper-Parameter Optimization

  2012cited by this paper
• How do humans sketch objects?

  2012cited by this paper
• What makes Paris look like Paris?

  2012influential reference
• Learning Feature Representations with K-Means

  2012cited by this paper
• Selecting Receptive Fields in Deep Networks

  2011cited by this paper
• ImageNet: A large-scale hierarchical image database

  2009cited by this paper
• Removing Camera Shake from a Single Photograph

  2009cited by this paper
• Sketch2Photo: internet image montage

  2009cited by this paper
• Supporting Online Material for Reducing the Dimensionality of Data with Neural Networks

  2006cited by this paper
• A non-local algorithm for image denoising

  2005cited by this paper
• Image quality assessment: from error visibility to structural similarity

  2004cited by this paper
• Image Analogies

  2001cited by this paper
• Image quilting for texture synthesis and transfer

  2001cited by this paper
• Color Transfer between Images

  2001cited by this paper
• Texture synthesis by non-parametric sampling

  1999cited by this paper

• Adapting Depth Anything to Adverse Imaging Conditions with Events

  2026cites this paper
• Trustworthy Longitudinal Brain MRI Completion: A Deformation-Based Approach with KAN-Enhanced Diffusion Model

  2026influential citation
• Toward Thermal Infrared Image Colorization via Large Kernel Convolution and Patch-Wise Graph Contrastive Learning

  2026cites this paper
• Face Normal Estimation from Rags to Riches

  2026influential citation
• From Parametric Determinism to Emergent Fusion: Data-Curated Style Control in Connectionist Architecture

  2026influential citation
• Comprehensive Machine Learning Benchmarking for Fringe Projection Profilometry with Photorealistic Synthetic Data

  2026cites this paper
• Few-Shot Specific Emitter Identification for Satellite IoE: A Novel Ground–Satellite Generative Learning Framework

  2026cites this paper
• An unsupervised image generation method for translation from fundus structure image to fluorescein angiography with fusion of hard exudates

  2026cites this paper
• Hybrid semantic segmentation with broad context and attention encoded network for urban street scenario

  2026cites this paper
• Forget Less by Learning Together through Concept Consolidation

  2026cites this paper
• Few-Shot LoRA Adaptation of a Flow-Matching Foundation Model for Cross-Spectral Object Detection

  2026cites this paper
• RoadMark-cGAN: Generative Conditional Learning to Directly Map Road Marking Lines from Aerial Orthophotos via Image-to-Image Translation

  2026influential citation
• 3D Wavelet-Based Structural Priors for Controlled Diffusion in Whole-Body Low-Dose PET Denoising

  2026influential citation
• Rescind: Countering Image Misconduct in Biomedical Publications with Vision-Language and State-Space Modeling

  2026cites this paper
• DADIGAN: A dual attention blocks-based disentangled iterative Generative Adversarial Network for cloud and shadow removal on SAR and optical images

  2026cites this paper
• Fine-Detailed Facial Sketch-to-Photo Synthesis With Detail-Enhanced Codebook Priors

  2026cites this paper
• Investigation of stochastic deep learning path planning methods for mobile robots

  2026cites this paper
• FGBVD-KD: Frequency-Guided bias-variance decomposition knowledge distillation for fracture detection

  2026cites this paper
• PPGSpeech: A Wearable Silent Speech Interface Leveraging Neck-Worn Photoplethysmography

  2026cites this paper
• End-to-end susceptibility-induced distortion correction for diffusion MRI with unsupervised deep learning

  2026cites this paper
• Pose Under Covers: In-Bed Human Pose Estimation Using Multisensor Image Fusion and Graph Convolutional Network

  2026cites this paper
• Improving Flexible Image Tokenizers for Autoregressive Image Generation

  2026cites this paper
• A Review of AI-Powered Controls in the Field of Magnetic Resonance Imaging

  2026cites this paper
• GeoDiff-SAR: A Geometric Prior Guided Diffusion Model for SAR Image Generation

  2026cites this paper
• SRU-Pix2Pix: A Fusion-Driven Generator Network for Medical Image Translation with Few-Shot Learning

  2026influential citation
• Edge-Assisted Domain Adaptive Passive RF-Based Sensing for Localization and Activity Recognition

  2026cites this paper
• Exploring generative artificial intelligence: a comprehensive guide

  2026cites this paper
• A novel RF-enabled Non-Destructive Inspection Method through Machine Learning and Programmable Wireless Environments

  2026cites this paper
• Visual-to-Tactile Cross-Modal Generation Using a Class-Conditional GAN with Multi-Scale Discriminator and Hybrid Loss

  2026cites this paper
• Sketch-Based Facade Renovation With Generative AI: A Streamlined Framework for Bypassing As-Built Modelling in Industrial Adaptive Reuse

  2026cites this paper
• A Survey on Deep Learning-Based Chinese Font Style Transfer

  2026cites this paper
• Weakly-Supervised Liver Lesion Detection in CT Images Using Adversarial Networks

  2026cites this paper
• Drim-NeRF: Diffusion-Based Restoration for Improving Neural Radiance Fields

  2026cites this paper
• DLPP-UL: Dose-level parameter prompted unpaired learning for low-dose CT image restoration

  2026cites this paper
• Contrastive and Dual Adversarial Representation Learning for Multi-View Clustering

  2026cites this paper
• An MoE-Driven Unified Image Restoration Framework for Adverse Weather Conditions

  2026cites this paper
• Self-Supervised Enhanced Imaging Network for Sparse Cross MIMO Arrays

  2026cites this paper
• Adaptive Fusion-Based Two-Flow Spectral–Spatial Transformer Network for SAR-Based Optical Cloud Removal

  2026cites this paper
• Noisy-label-adaptive SegFormer parameter optimization for high-enthalpy flow PLIF image segmentation

  2026cites this paper
• SMHNet: Self-Supervised Multiscale Hierarchical Network for High Fidelity 3-D Face Reconstruction

  2026influential citation
• WDUIE: Wavelet Decoupling Network for Underwater Image Enhancement

  2026cites this paper
• Comparison of Conventional Machine Learning and Convolutional Deep Learning Models for Seagrass Mapping Using Satellite Imagery

  2026cites this paper
• Multimodal 3-D Target Reconstruction From SAR and Optical Imagery via Differentiable Rendering

  2026cites this paper
• TorDet: A Refined Two-Stage Deep Learning Approach for Radar-Based Tornado Detection

  2026cites this paper
• Implicit neural representations for surrogate modeling in the built environment

  2026cites this paper
• Sustainable Steganography via Neural Trojans for Geographic Satellite Remote Sensing

  2026cites this paper
• A two-stage deep learning method for predicting turbine vane temperature fields under active cooling air flow modulation

  2026cites this paper
• Multi-Constraint Physics-Informed Generative Adversarial Networks (PIGAN) Enable Small-Data Learning for Urban Wind Field Prediction

  2026cites this paper
• RGB-to-NIR Facial Image Translation via Lightweight Generative Adversarial Network

  2026cites this paper
• Deepfake detection: critical review of state-of-the-art approaches and future perspectives

  2026influential citation
• Re-Align: Structured Reasoning-guided Alignment for In-Context Image Generation and Editing

  2026cites this paper
• Multi-Scale Local Speculative Decoding for Image Generation

  2026cites this paper
• A Hierarchical Multi-Resolution Self-Supervised Framework for High-Fidelity 3D Face Reconstruction Using Learnable Gabor-Aware Texture Modeling

  2026cites this paper
• Workflow for Volumetric Uncertainty and Sensitivity Analysis of Machine Learning‐Interpreted Seismic Horizons at the Smeaheia CO2 Project Site

  2026cites this paper
• A synthetic-data-trained deep learning model for predicting residual stress and melt pool characteristics in laser melting with arbitrary beam profiles

  2026cites this paper
• GDiT: A graph-prior-guided diffusion transformer for semantic-controllable remote sensing image synthesis

  2026cites this paper
• A multi-scale skip feature fusion underwater image enhancement method based on cycle adversarial network

  2026cites this paper
• Synthetic FMCW Radar Range Azimuth Maps Augmentation with Generative Diffusion Model

  2026cites this paper
• OSCAR: Optical-aware Semantic Control for Aleatoric Refinement in Sar-to-Optical Translation

  2026cites this paper
• Bridging Robustness and Efficiency: Real-Time Low-Light Enhancement via Attention U-Net GAN

  2026cites this paper
• UIT-GAN: A cross-modal generative model for ultrasonic infrared thermography detection of impact damage in composites

  2026cites this paper
• Unified Multi-Site Multi-Sequence Brain MRI Harmonization Enriched by Biomedical Semantic Style

  2026cites this paper
• Image SNR Enhancement for a Short Axial FOV Brain PET System Using Generative Deep Learning

  2026influential citation
• Diffusion-Based Model for Parametric Ki Generation From Total-Body Dynamic PET of Short-Duration Scan

  2026cites this paper
• Distributed Generative Model: A Data Synthesizing Framework for Multisource Heterogeneous Data

  2026cites this paper
• A Self-Supervised Foundation Model for Robust and Generalizable Representation Learning in STED Microscopy

  2026cites this paper
• SemiSketch: An ancient mural sketch extraction network based on reference prior and gradient frequency compensation

  2026cites this paper
• Domain Adaptation With Generated Infrared Images for Multi-Spectra Pedestrian Detection

  2026cites this paper
• CollaboRadio: A Hybrid Device-Edge-Cloud Collaboration Paradigm for Fine-Grained Radio Map Construction

  2026cites this paper
• Weakly Supervised Image Dehazing via Physics-Based Decomposition

  2026cites this paper
• Annealing Genetic Slicing Adversarial Networks Based Feedback for Imbalanced Visual Classification

  2026cites this paper
• A comprehensive survey on image fusion: Which approach fits which need

  2026cites this paper
• From discrete to continuous: A spatiotemporal evolution-aware adversarial diffusion framework for retinal disease progression prediction

  2026cites this paper
• Cross-domain knowledge integration framework for optical coherence tomography images denoising

  2026cites this paper
• Vision Image-Aided Near-Field Beam Training for Internet of Vehicles Communication Systems: From Daytime to Nighttime

  2026cites this paper
• Deep learning-driven virtual phase shifting for enhanced interferometric surface profiling

  2026cites this paper
• A review of instruction-guided image editing

  2026cites this paper
• Multi-condition guided unpaired MRI T1-to-FLAIR image-to-image translation

  2026cites this paper
• Generic-to-Personalised Learning for Multimodal Image Synthesis With Bidirectional Variational GAN

  2026cites this paper
• A multi-resolution Feature fusion network for significant wave height downscaling

  2026cites this paper
• Animating Faces With Emotions Through a Generative Adversarial Network Preserving Identity

  2026cites this paper
• Generative AI for spatial regeneration planning: Integrating urban planning theories and ethics

  2026cites this paper
• Virtual dispersion and gradation prediction of stacked particles via improved Pix2Pix and SAM framework

  2026cites this paper
• Generative models for SAR-optical image translation: A systematic review

  2026cites this paper
• Multiview Hand Gesture Recognition From Generated Gestures Using Conditional Adversarial Network

  2026cites this paper
• Diffusion-enhanced semantic segmentation for underground crack detection

  2026cites this paper
• PowerEar: An Audio Eavesdropping Attack on Mobile Devices Through USB Power Side Channel

  2026cites this paper
• Controllable image-Guided generation via dynamic gaussian spectral modulation

  2026cites this paper
• GAN4RM: A CWGAN-Based Framework for Radio Maps Generation in Real Cellular Networks

  2026cites this paper
• Hyperspectral Image Synthesis Through Blind Unmixing Dictionary and Deep Diffusion Models

  2026cites this paper
• Deep Learning Segmentation Methods for Mandibular Defects From Computed Tomography Images: A Review

  2026cites this paper
• Winter damage diagnostic modeling based on synthetic vegetation indices from UAV-based multispectral imaging

  2026cites this paper
• Improved Conditional GAN-Based Channel Estimation for MIMO-OFDM Systems

  2026influential citation
• Fast simulation of the deformed foundation pit using graph neural networks

  2026cites this paper
• HVTC-GAN: A High-Level Vision Task Cooperative GAN for SAR-to-Optical Translation via Semantic Segmentation

  2026cites this paper
• Progressive temporal compensation and semantic enhancement for Exo-to-Ego video generation

  2026cites this paper
• Image enhancement for accelerated MRI using a joint GAN and diffusion model framework

  2026cites this paper
• Topology-aware Pathological Consistency Matching for Weakly-Paired IHC Virtual Staining

  2026cites this paper
• Prior-oriented specific and triple-view general prompts for multi-weather degraded image restoration

  2026cites this paper
• Sim2real Image Translation Enables Viewpoint-Robust Policies from Fixed-Camera Datasets

  2026cites this paper