PCANet-Based Structural Representation for Nonrigid Multimodal Medical Image Registration

Nonrigid multimodal image registration remains a challenging task in medical image processing and analysis. The structural representation (SR)-based registration methods have attracted much attention recently. However, the existing SR methods cannot provide satisfactory registration accuracy due to the utilization of hand-designed features for structural representation. To address this problem, the structural representation method based on the improved version of the simple deep learning network named PCANet is proposed for medical image registration. In the proposed method, PCANet is firstly trained on numerous medical images to learn convolution kernels for this network. Then, a pair of input medical images to be registered is processed by the learned PCANet. The features extracted by various layers in the PCANet are fused to produce multilevel features. The structural representation images are constructed for two input images based on nonlinear transformation of these multilevel features. The Euclidean distance between structural representation images is calculated and used as the similarity metrics. The objective function defined by the similarity metrics is optimized by L-BFGS method to obtain parameters of the free-form deformation (FFD) model. Extensive experiments on simulated and real multimodal image datasets show that compared with the state-of-the-art registration methods, such as modality-independent neighborhood descriptor (MIND), normalized mutual information (NMI), Weber local descriptor (WLD), and the sum of squared differences on entropy images (ESSD), the proposed method provides better registration performance in terms of target registration error (TRE) and subjective human vision.

• Publication year

  2018

• Venue

  Italian National Conference on Sensors

• Publication date

  2018-05-01

• Fields of study

  Medicine, Computer Science

• Identifiers
  DOI 10.3390/s18051477PMID 29738512PMCID 5982469
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network

  2017cited by this paper
• Multimodal Image Registration with Deep Context Reinforcement Learning

  2017cited by this paper
• Label-driven weakly-supervised learning for multimodal deformarle image registration

  2017cited by this paper
• Deformable Image Registration Based on Similarity-Steered CNN Regression

  2017cited by this paper
• Nonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks

  2017cited by this paper
• Richer Convolutional Features for Edge Detection

  2016cited by this paper
• Scalable High-Performance Image Registration Framework by Unsupervised Deep Feature Representations Learning

  2016cited by this paper
• An excellent navigation system and experience in craniomaxillofacial navigation surgery: a double-center study

  2016cited by this paper
• Vascular Structure Identification in Intraoperative 3D Contrast-Enhanced Ultrasound Data

  2016cited by this paper
• An Artificial Agent for Robust Image Registration

  2016cited by this paper
• Learning intervention-induced deformations for non-rigid MR-CT registration and electrode localization in epilepsy patients

  2015cited by this paper
• Structured Decision Forests for Multi-modal Ultrasound Image Registration

  2015cited by this paper
• PCANet: A Simple Deep Learning Baseline for Image Classification?

  2014cited by this paper
• Diffusion Maps for Multimodal Registration

  2014cited by this paper
• Self-similarity weighted mutual information: A new nonrigid image registration metric

  2014cited by this paper
• P-spline based nonrigid brain MR image registration using regional mutual information

  2013cited by this paper
• Two Phase Non-Rigid Multi-Modal Image Registration Using Weber Local Descriptor-Based Similarity Metrics and Normalized Mutual Information

  2013cited by this paper
• Multi-Modal Image Registration Based on Gradient Orientations of Minimal Uncertainty

  2012cited by this paper
• Deformable multimodal registration with gradient orientation based on structure tensors

  2012cited by this paper
• Self-similarity Weighted Mutual Information: A New Nonrigid Image Registration Metric

  2012cited by this paper
• Entropy and Laplacian images: Structural representations for multi-modal registration

  2012cited by this paper
• MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration

  2012cited by this paper
• Clinical application of a 3D ultrasound-guided prostate biopsy system.

  2011cited by this paper
• Remark on “algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound constrained optimization”

  2011cited by this paper
• Nonrigid Image Registration Using Conditional Mutual Information

  2010cited by this paper
• GRAM: A framework for geodesic registration on anatomical manifolds

  2010cited by this paper
• Real-time MRI-TRUS fusion for guidance of targeted prostate biopsies

  2008cited by this paper
• Non-rigid registration of multi-modal images using both mutual information and cross-correlation

  2008cited by this paper
• Intensity Gradient Based Registration and Fusion of Multi-modal Images

  2007cited by this paper
• Evaluation of the effects of Gabor filter parameters on texture classification

  2007cited by this paper
• Intensity Gradient Based Registration and Fusion of Multi-modal Images

  2006cited by this paper
• Normalized mutual information based registration using k-means clustering and shading correction

  2006influential reference
• Intensity Robust Viscous Fluid Deformation Based Morphometry Using Regionally Adapted Mutual Information

  2005cited by this paper
• A non-local algorithm for image denoising

  2005cited by this paper
• Nonrigid registration using free-form deformations: application to breast MR images

  1999cited by this paper
• Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization

  1997cited by this paper
• Alignment by Maximization of Mutual Information

  1995cited by this paper

• Robust multi-modal COVID-19 medical image registration using dense deep learning descriptor model

  2025cites this paper
• ACSwinNet: A Deep Learning-Based Rigid Registration Method for Head-Neck CT-CBCT Images in Image-Guided Radiotherapy

  2024cites this paper
• Unsupervised registration for liver CT-MR images based on the multiscale integrated spatial-weight module and dual similarity guidance

  2023cites this paper
• Free form deformation and symmetry constraint-based multi-modal brain image registration using generative adversarial nets

  2023cites this paper
• Symmetry-based representation for registration of multimodal images

  2022cites this paper
• Towards Computing Cross-modality Symmetric Non-Rigid Medical Image Registration

  2022cites this paper
• A review of deep learning-based deformable medical image registration

  2022cites this paper
• Non-rigid multi-modal brain image registration based on two-stage generative adversarial nets

  2022cites this paper
• Deep Learning for Medical Image Registration: A Comprehensive Review

  2022cites this paper
• Regional Localization of Mouse Brain Slices Based on Unified Modal Transformation

  2021cites this paper
• Deep learning based data-adaptive descriptor for non-rigid multi-modal medical image registration

  2021cites this paper
• Hierarchical Registration of Brain Images Based on B-splines and Laplacian Commutators

  2021cites this paper
• LAPNet: Non-Rigid Registration Derived in k-Space for Magnetic Resonance Imaging

  2021cites this paper
• Modality-agnostic self-supervised deep feature learning and fast instance optimisation for multimodal fusion in ultrasound-guided interventions

  2021cites this paper
• Multi-modal image registration in the presence of spatially varying intensity distortion using structural representation

  2021cites this paper
• General discriminative optimization for point set registration

  2021cites this paper
• An Unsupervised Convolution Neural Network for Deformable Registration of Mono/Multi-Modality Medical Images

  2021cites this paper
• Self-learning based medical image representation for rigid real-time and multimodal slice-to-volume registration

  2020cites this paper
• Medical Image Registration Using Deep Neural Networks: A Comprehensive Review

  2020cites this paper
• Artificial intelligence in radiotherapy: a technological review

  2020cites this paper
• SSD-TSEFFM: New SSD Using Trident Feature and Squeeze and Extraction Feature Fusion

  2020cites this paper
• A semi-supervised convolutional neural network based on subspace representation for image classification

  2020cites this paper
• Boosting Structure Consistency for Multispectral and Multimodal Image Registration

  2020cites this paper
• Machine learning for radiomics-based multi-modality and multi-parametric modeling.

  2019cites this paper
• A Review of Point Set Registration: From Pairwise Registration to Groupwise Registration

  2019cites this paper
• Non-Rigid Multi-Modal 3D Medical Image Registration Based on Foveated Modality Independent Neighborhood Descriptor

  2019cites this paper
• Implementation and Validation of a Three-dimensional Cardiac Motion Estimation Network.

  2019cites this paper