Image completion with structure propagation

In this paper, we introduce a novel approach to image completion, which we call structure propagation. In our system, the user manually specifies important missing structure information by extending a few curves or line segments from the known to the unknown regions. Our approach synthesizes image patches along these user-specified curves in the unknown region using patches selected around the curves in the known region. Structure propagation is formulated as a global optimization problem by enforcing structure and consistency constraints. If only a single curve is specified, structure propagation is solved using Dynamic Programming. When multiple intersecting curves are specified, we adopt the Belief Propagation algorithm to find the optimal patches. After completing structure propagation, we fill in the remaining unknown regions using patch-based texture synthesis. We show that our approach works well on a number of examples that are challenging to state-of-the-art techniques.

• Publication year

  2005

• Venue

  ACM Transactions on Graphics

• Publication date

  2005-07-01

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1145/1186822.1073274
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Context-based surface completion

  2004cited by this paper
• PatchWorks: Example-Based Region Tiling for Image Editing

  2004cited by this paper
• Digital image inpainting using Monte Carlo method

  2004cited by this paper
• Object removal by exemplar-based inpainting

  2003influential reference
• Understanding belief propagation and its generalizations

  2003cited by this paper
• Graphcut textures: image and video synthesis using graph cuts

  2003cited by this paper
• Fragment-based image completion

  2003influential reference
• Poisson image editing

  2003cited by this paper
• Simultaneous structure and texture image inpainting

  2003cited by this paper
• Learning how to inpaint from global image statistics

  2003cited by this paper
• Image repairing: robust image synthesis by adaptive ND tensor voting

  2003cited by this paper
• Missing data correction in still images and image sequences

  2002cited by this paper
• Object-based image editing

  2002cited by this paper
• Stereo Matching Using Belief Propagation

  2002cited by this paper
• Navier-Stokes, Fluid Dynamics, and Image and Video Inpainting

  2001cited by this paper
• Synthesizing natural textures

  2001cited by this paper
• Navier-stokes, fluid dynamics, and image and video inpainting

  2001cited by this paper
• Image Analogies

  2001cited by this paper
• On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs

  2001cited by this paper
• Image quilting for texture synthesis and transfer

  2001cited by this paper
• A Non-Hierarchical Procedure for Re-Synthesis of Complex Textures

  2001cited by this paper
• Real-time texture synthesis by patch-based sampling

  2001cited by this paper
• Filling-in by joint interpolation of vector fields and gray levels

  2001cited by this paper
• Nontexture Inpainting by Curvature-Driven Diffusions

  2001cited by this paper
• Image inpainting

  2000cited by this paper
• Fast texture synthesis using tree-structured vector quantization

  2000cited by this paper
• Texture synthesis by non-parametric sampling

  1999cited by this paper
• Learning Low-Level Vision

  1999influential reference
• Finding out about filling-in: a guide to perceptual completion for visual science and the philosophy of perception.

  1998cited by this paper
• Image replacement through texture synthesis

  1997cited by this paper
• Probabilistic reasoning in intelligent systems - networks of plausible inference

  1989cited by this paper
• Chapter 2 – BAYESIAN INFERENCE

  1988cited by this paper
• Dynamic Programming

  1957cited by this paper
• Principles Of Gestalt Psychology

  1936cited by this paper
• Transactions on Pattern Analysis and Machine Intelligence 1 Space-time Video Completion Draft Transactions on Pattern Analysis and Machine Intelligence 2

  year unknowncited by this paper

• Optimizing Paths for Adaptive Fly-Scan Microscopy: An Extended Version

  2025cites this paper
• Mask Optimization with Auxiliary Lines for Image Inpainting

  2025influential citation
• Image Completion Using Automatic Structure Propagation With Bézier Curves

  2025cites this paper
• U-Paint: Image Inpainting and Object Detection via Partial Convolutions

  2025cites this paper
• HRC-Net: Learning Visual Hypothesis, Representative, and Collaboration for Multi-Domain Image Inpainting

  2025cites this paper
• Image inpainting via Multi-scale Adaptive Priors

  2025cites this paper
• A fish school feeding intensity assessment method based on EfficientNetB0-FPSANet

  2025cites this paper
• Recovering Partially Corrupted Objects via Sketch-Guided Bidirectional Feature Interaction

  2025cites this paper
• WFIL-NET: image inpainting based on wavelet downsampling and frequency integrated learning module

  2024cites this paper
• Delving Into Important Samples of Semi-Supervised Old Photo Restoration: A New Dataset and Method

  2024cites this paper
• Structural self-attention GAN-based inpainting of high dynamic range fringe pattern for 3D measurement of metal gear teeth

  2024cites this paper
• Don't Look into the Dark: Latent Codes for Pluralistic Image Inpainting

  2024cites this paper
• Weighted Similarity-Confidence Laplacian Synthesis for High-Resolution Art Painting Completion

  2024cites this paper
• Building Facade-Completion Network Based on Dynamic Convolutional GAN

  2024cites this paper
• Symmetric Connected U-Net with Multi-Head Self Attention (MHSA) and WGAN for Image Inpainting

  2024cites this paper
• Face Mask Removal with Region-attentive Face Inpainting

  2024cites this paper
• Model-Guided Generative Adversarial Networks for Unsupervised Fine-Grained Image Generation

  2024cites this paper
• CORE-MPI: Consistency Object Removal with Embedding MultiPlane Image

  2024cites this paper
• Cascaded refinement residual attention network for image outpainting

  2024cites this paper
• CCA: collaborative competitive agents for image editing

  2024cites this paper
• Self-prior guided generative adversarial network for image inpainting

  2024cites this paper
• Image Inpainting Using Deep Learning Techniques: A Review

  2023cites this paper
• Toward Interactive Image Inpainting via Robust Sketch Refinement

  2023cites this paper
• Object Removing via Multi-Column Gate Convolutional Neutral Network

  2023cites this paper
• The Evaluation of the Grade of Leaf Disease in Apple Trees Based on PCA-Logistic Regression Analysis

  2023cites this paper
• Structure-Texture Consistent Painting Completion for Artworks

  2023influential citation
• Image Inpainting Using PatchGAN

  2023cites this paper
• Deep generative network for image inpainting with gradient semantics and spatial-smooth attention

  2023cites this paper
• High resolution Mural images inpainting based on Dual Aggregation Generative Adversarial Network

  2023cites this paper
• Reference-Guided Large-Scale Face Inpainting With Identity and Texture Control

  2023cites this paper
• The Evaluation of the Grade of Leaf Disease in Apple Trees Based on PCA-Logistic Regression Analysis

  2023cites this paper
• Multi-stage Progressive Reasoning for Dunhuang Murals Inpainting

  2023cites this paper
• A weakly supervised inpainting-based learning method for lung CT image segmentation

  2023cites this paper
• Towards Interactive Image Inpainting via Sketch Refinement

  2023cites this paper
• Image Inpainting using Automatic Structure Propagation with Auxiliary Line Construction

  2022influential citation
• Image Inpainting via Iteratively Decoupled Probabilistic Modeling

  2022cites this paper
• Instance-Aware Image Completion

  2022cites this paper
• Efficient Generative Face Completion with Perceptual Deblurring

  2022cites this paper
• Context-Aware Image Completion

  2022cites this paper
• SDM: Spatial Diffusion Model for Large Hole Image Inpainting

  2022cites this paper
• The Family of Onion Convolutions for Image Inpainting

  2022cites this paper
• Overview of Image Inpainting Techniques: A Survey

  2022cites this paper
• Image Retrieval Using Digital Image Inpainting Techniques

  2022cites this paper
• SIFNet: Free-form image inpainting using color split-inpaint-fuse approach

  2022cites this paper
• Reference-Guided Landmark Image Inpainting With Deep Feature Matching

  2022cites this paper
• Hybrid texture synthesis and interpolated structure image completion

  2022cites this paper
• An Analysis of Generative Methods for Multiple Image Inpainting

  2022cites this paper
• MAT: Mask-Aware Transformer for Large Hole Image Inpainting

  2022cites this paper
• Research on hair removal algorithm of dermatoscopic images based on maximum variance fuzzy clustering and optimization Criminisi algorithm

  2022cites this paper
• Image inpainting based on deep learning: A review

  2022cites this paper
• Continuous digital zooming using generative adversarial networks for dual camera system

  2021cites this paper
• Open World Entity Segmentation

  2021cites this paper
• A Survey of Human-in-the-loop for Machine Learning

  2021cites this paper
• Feature learning and patch matching for diverse image inpainting

  2021cites this paper
• Digital Image Inpainting Techniques for Cultural Heritage Preservation and Restoration

  2021cites this paper
• Image inpainting based on deep learning: A review

  2021cites this paper
• Image Inpainting by End-to-End Cascaded Refinement With Mask Awareness

  2021cites this paper
• Image completion with approximate convex hull tensor decomposition

  2021cites this paper
• Layout-Guided Novel View Synthesis from a Single Indoor Panorama

  2021cites this paper
• A comprehensive review of past and present image inpainting methods

  2021cites this paper
• E2I: Generative Inpainting From Edge to Image

  2021influential citation
• Image Inpainting with Global Optimization using Parallel Computing

  2021cites this paper
• Aggregated Contextual Transformations for High-Resolution Image Inpainting

  2021cites this paper
• Spatio-spectral Image Reconstruction Using Non-local Filtering

  2021cites this paper
• Rectangling Irregular Videos by Optimizing Spatio-Temporal Warping

  2021cites this paper
• Towards Full-to-Empty Room Generation with Structure-Aware Feature Encoding and Soft Semantic Region-Adaptive Normalization

  2021cites this paper
• Uncertainty-Aware Semantic Guidance and Estimation for Image Inpainting

  2021cites this paper
• Controllable Sketch-to-Image Translation for Robust Face Synthesis

  2021cites this paper
• Rectangling irregular videos by optimal spatio-temporal warping

  2021cites this paper
• Image quality assessment based on manifold distortion Manifold bozulması ile imge

  2021cites this paper
• Interactive Image Inpainting of Large-Scale Missing Region

  2021influential citation
• Fully Context-Aware Image Inpainting with a Learned Semantic Pyramid

  2021cites this paper
• Learning Contextual Transformer Network for Image Inpainting

  2021cites this paper
• Divergence-Based Adaptive Extreme Video Completion

  2020cites this paper
• Research on Region Selection Strategy for Visible Watermark Embedding

  2020cites this paper
• Deep Neural Networks for In Situ Hybridization Grid Completion and Clustering

  2020cites this paper
• Image Extrapolation Based on Perceptual Loss and Style Loss

  2020cites this paper
• Image Inpainting Based on Inside–Outside Attention and Wavelet Decomposition

  2020cites this paper
• VCNet: A Robust Approach to Blind Image Inpainting

  2020cites this paper
• Image Inpainting with Onion Convolutions

  2020cites this paper
• An Interactive Perspective Scene Completion Framework Guided by Complanate Mesh

  2020cites this paper
• Draw with me: human-in-the-loop for image restoration

  2020cites this paper
• Image Restoration using Mirroring Method Which Based on the Gradient Direction

  2020cites this paper
• Instance Separation Emerges from Inpainting

  2020cites this paper
• Image Completion with Hybrid Interpolation in Tensor Representation

  2020cites this paper
• Symmetric Skip Connection Wasserstein GAN for High-Resolution Facial Image Inpainting

  2020cites this paper
• DATA PRIORS IN VIDEOS AND IMAGES

  2020cites this paper
• Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketches

  2020cites this paper
• Text-Guided Image Inpainting

  2020cites this paper
• User-Guided Chinese Painting Completion–A Generative Adversarial Network Approach

  2020cites this paper
• iOrthoPredictor

  2020cites this paper
• Graph Laplacian for spectral clustering and seeded image segmentation

  2020cites this paper
• Point Cloud Scene Completion of Obstructed Building Facades with Generative Adversarial Inpainting

  2020cites this paper
• Inpainting Networks Learn to Separate Cells in Microscopy Images

  2020cites this paper
• Deep Generative Model for Image Inpainting With Local Binary Pattern Learning and Spatial Attention

  2020cites this paper
• Object Removal and Image Restoration within Subspaces by Prioritized Patch Optimization

  2020cites this paper
• Global and Local Attention-Based Free-Form Image Inpainting

  2020cites this paper
• A Survey on Patch-based Synthesis: GPU Implementation and Optimization

  2020cites this paper
• Attentional coarse-and-fine generative adversarial networks for image inpainting

  2020cites this paper
• Image completion via transformation and structural constraints

  2020cites this paper