Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs

We present a deep convolutional decoder architecture that can generate volumetric 3D outputs in a compute- and memory-efficient manner by using an octree representation. The network learns to predict both the structure of the octree, and the occupancy values of individual cells. This makes it a particularly valuable technique for generating 3D shapes. In contrast to standard decoders acting on regular voxel grids, the architecture does not have cubic complexity. This allows representing much higher resolution outputs with a limited memory budget. We demonstrate this in several application domains, including 3D convolutional autoencoders, generation of objects and whole scenes from high-level representations, and shape from a single image.

• Publication year

  2017

• Venue

  IEEE International Conference on Computer Vision

• Publication date

  2017-03-28

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1109/ICCV.2017.230arXiv 1703.09438
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• OctNet: Learning Deep 3D Representations at High Resolutions

  2016cited by this paper
• Perceptual Losses for Real-Time Style Transfer and Super-Resolution

  2016cited by this paper
• Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision

  2016cited by this paper
• 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation

  2016cited by this paper
• Learning Shape Abstractions by Assembling Volumetric Primitives

  2016cited by this paper
• 3D Shape Induction from 2D Views of Multiple Objects

  2016cited by this paper
• VConv-DAE: Deep Volumetric Shape Learning Without Object Labels

  2016cited by this paper
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

  2016cited by this paper
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

  2016cited by this paper
• Unsupervised Learning of 3D Structure from Images

  2016cited by this paper
• 3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction

  2016cited by this paper
• Learning a Predictable and Generative Vector Representation for Objects

  2016cited by this paper
• A Point Set Generation Network for 3D Object Reconstruction from a Single Image

  2016cited by this paper
• Deep Learning 3D Shape Surfaces Using Geometry Images

  2016cited by this paper
• SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation

  2016cited by this paper
• Deep disentangled representations for volumetric reconstruction

  2016cited by this paper
• Deep Shape from a Low Number of Silhouettes

  2016cited by this paper
• Learning Semantic Deformation Flows with 3D Convolutional Networks

  2016cited by this paper
• VoxResNet: Deep Voxelwise Residual Networks for Volumetric Brain Segmentation

  2016cited by this paper
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

  2015cited by this paper
• FlowNet: Learning Optical Flow with Convolutional Networks

  2015cited by this paper
• Sparse 3D convolutional neural networks

  2015cited by this paper
• ShapeNet: An Information-Rich 3D Model Repository

  2015cited by this paper
• Learning Deconvolution Network for Semantic Segmentation

  2015cited by this paper
• Global, Dense Multiscale Reconstruction for a Billion Points

  2015cited by this paper
• Geodesic Convolutional Neural Networks on Riemannian Manifolds

  2015cited by this paper
• Multi-view 3D Models from Single Images with a Convolutional Network

  2015cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• Learning to Generate Chairs, Tables and Cars with Convolutional Networks

  2014cited by this paper
• Volumetric 3D mapping in real-time on a CPU

  2014cited by this paper
• Caffe: Convolutional Architecture for Fast Feature Embedding

  2014cited by this paper
• Spatially-sparse convolutional neural networks

  2014cited by this paper
• FAUST: Dataset and Evaluation for 3D Mesh Registration

  2014cited by this paper
• 3D ShapeNets: A deep representation for volumetric shapes

  2014cited by this paper
• SSD: Smooth Signed Distance Surface Reconstruction

  2011cited by this paper
• Fusion of depth maps with multiple scales

  2011cited by this paper
• Poisson surface reconstruction

  2006cited by this paper
• Cumulative generation of octree models from range data

  1984cited by this paper
• Linear octtrees for fast processing of three-dimensional objects

  1982cited by this paper

• DCCVT: Differentiable Clipped Centroidal Voronoi Tessellation

  2026cites this paper
• Spatial location and distribution reconstruction of the leaking gas plume via a single infrared remote sensing system.

  2026cites this paper
• Compression and Ray-March Rendering Using Implicit Neural Representations of Data Values and Domain Extent

  2026cites this paper
• CMNet: Cross-Modal Coarse-to-Fine Network for Point Cloud Completion Based on Patches

  2025cites this paper
• RayletDF: Raylet Distance Fields for Generalizable 3D Surface Reconstruction from Point Clouds or Gaussians

  2025cites this paper
• Self-Balancing, Memory Efficient, Dynamic Metric Space Data Maintenance, for Rapid Multi-Kernel Estimation

  2025cites this paper
• 3D-Aware Image Synthesis via Local Spatial Self-Attention

  2025cites this paper
• 3D Shape Generation: A Survey

  2025cites this paper
• D-LaMa: Depth Inpainting of Perspective-Occluded Environments

  2025cites this paper
• Evaluating Latent Generative Paradigms for High-Fidelity 3D Shape Completion from a Single Depth Image

  2025cites this paper
• A Comprehensive Survey on 3D Single-View Object Reconstruction

  2025cites this paper
• Efficient 3D Perception on Embedded Systems via Interpolation-Free Tri-Plane Lifting and Volume Fusion

  2025cites this paper
• Co-Adaptive Neural Radiance Field With Geometry-Feature

  2025cites this paper
• Learning implicit representations from hierarchical structures for single-view 3D reconstruction

  2025cites this paper
• NoKSR: Kernel-Free Neural Surface Reconstruction via Point Cloud Serialization

  2025cites this paper
• Generation driven understanding of localized 3D scenes with 3D diffusion model

  2025cites this paper
• Bolt3D: Generating 3D Scenes in Seconds

  2025cites this paper
• $\mathrm{Tri^{2}plane}$: Advancing Neural Implicit Surface Reconstruction for Indoor Scenes

  2025cites this paper
• A 3D Generation Framework from Cross Modality to Parameterized Primitive

  2025cites this paper
• 基于深度学习的三维重建方法综述

  2025cites this paper
• Deep learning for three-dimensional (3D) plant phenomics

  2025cites this paper
• Deep Retraining Approach for Category-Specific 3D Reconstruction Models from a Single 2D Image

  2025cites this paper
• PartSDF: Part-Based Implicit Neural Representation for Composite 3D Shape Parametrization and Optimization

  2025cites this paper
• Optimized generative adversarial network for efficient resolution enhancement of 3D segmented rock tomography

  2025cites this paper
• Process, challenges and solutions of fruit 3D reconstruction: A review

  2025cites this paper
• Learning to Predict 3D Meshes from a Single Image via Depth Consistency

  2025cites this paper
• Transformers Based 3D Image Reconstruction Using 2D Low Resolution Image Data

  2025cites this paper
• Learning Neural Velocity Fields from Dynamic 3D Scenes via Inter-Frame Consistency Loss

  2025cites this paper
• Multispectral-NeRF:a multispectral modeling approach based on neural radiance fields

  2025cites this paper
• StrucADT: Generating Structure-Controlled 3D Point Clouds With Adjacency Diffusion Transformer

  2025cites this paper
• LEO-VL: Efficient Scene Representation for Scalable 3D Vision-Language Learning

  2025cites this paper
• Single-view 3D reconstruction of surface of revolution

  2025cites this paper
• Robust novel view synthesis from multi-view feature stereo matching priors

  2025cites this paper
• MS-GAN: 3D deep generative model for multi-species propeller parameterization and generation

  2025cites this paper
• Learning Neural Velocity Fields from Dynamic 3D Scenes via Edge-Aware Ray Sampling

  2025cites this paper
• Unified Cross-Scale 3D Generation and Understanding via Autoregressive Modeling

  2025cites this paper
• Efficient Autoregressive Shape Generation via Octree-Based Adaptive Tokenization

  2025cites this paper
• Patch-Grid: An Efficient and Feature-Preserving Neural Implicit Surface Representation

  2025cites this paper
• Memory-Efficient Super-Resolution of 3D Micro-CT Images Using Octree-Based GANs: Enhancing Resolution and Segmentation Accuracy

  2025cites this paper
• 3D estimation of single-view 2d images using shape priors and transfer learning

  2024cites this paper
• Machine and Deep Learning Implementations for Heritage Building Information Modelling: A Critical Review of Theoretical and Applied Research

  2024cites this paper
• HD-VoxelFlex: Flexible High-Definition Voxel Grid Representation

  2024cites this paper
• Enhancing Point Cloud Resolution for Autonomous Driving with Deep Learning AI Models

  2024cites this paper
• Unsupervised Occupancy Learning from Sparse Point Cloud

  2024cites this paper
• PointCompress3D: A Point Cloud Compression Framework for Roadside LiDARs in Intelligent Transportation Systems

  2024cites this paper
• Multi-view 3D reconstruction based on deep learning: A survey and comparison of methods

  2024cites this paper
• Shape generation via learning an adaptive multimodal prior

  2024cites this paper
• Sparse Convolutional Networks for Surface Reconstruction from Noisy Point Clouds

  2024cites this paper
• Fully Automatic Karyotyping via Deep Convolutional Neural Networks

  2024cites this paper
• 基于改进PointNet++的Lidar点云分割模型

  2024cites this paper
• LATTE3D: Large-scale Amortized Text-To-Enhanced3D Synthesis

  2024cites this paper
• CVT-xRF: Contrastive In-Voxel Transformer for 3D Consistent Radiance Fields from Sparse Inputs

  2024cites this paper
• APC2Mesh: Bridging the gap from occluded building fa\c{c}ades to full 3D models

  2024cites this paper
• CMFAN: Cross-Modal Feature Alignment Network for Few-Shot Single-View 3D Reconstruction

  2024cites this paper
• Accurate Training Data for Occupancy Map Prediction in Automated Driving Using Evidence Theory

  2024cites this paper
• Bayesian Mesh Optimization for Graph Neural Networks to Enhance Engineering Performance Prediction

  2024cites this paper
• 3D Shape Completion on Unseen Categories: A Weakly-Supervised Approach

  2024cites this paper
• Everted U-Net for 3D scene reconstruction and segmentation

  2024cites this paper
• 3D Reconstruction and Novel View Synthesis of Indoor Environments Based on a Dual Neural Radiance Field

  2024cites this paper
• PushNet: 3D reconstruction from a single image by pushing

  2024cites this paper
• Point-Reconstruction-Network: 3D Point Cloud Reconstruction from a Single Image

  2024cites this paper
• Rapid Abstraction of Spacecraft 3D Structure from Single 2D Image

  2024cites this paper
• VIPNet: Combining Viewpoint Information and Shape Priors for Instant Multi-view 3D Reconstruction

  2024cites this paper
• Multi-Head Attention Refiner for Multi-View 3D Reconstruction

  2024cites this paper
• Don't Mesh with Me: Generating Constructive Solid Geometry Instead of Meshes by Fine-Tuning a Code-Generation LLM

  2024cites this paper
• L3DG: Latent 3D Gaussian Diffusion

  2024cites this paper
• RAD: A Dataset and Benchmark for Real-Life Anomaly Detection with Robotic Observations

  2024cites this paper
• 3D Representation Methods: A Survey

  2024cites this paper
• PixelGaussian: Generalizable 3D Gaussian Reconstruction from Arbitrary Views

  2024cites this paper
• PesRec: A parametric estimation method for indoor semantic scene reconstruction from a single image

  2024cites this paper
• Deep Learning at the Intersection: Certified Robustness as a Tool for 3D Vision

  2024cites this paper
• Airvox: efficient computational fluid dynamics prediction using 3D convolutional neural networks for building design

  2024cites this paper
• GALA: Geometry-Aware Local Adaptive Grids for Detailed 3D Generation

  2024influential citation
• Optimizing 3D Geometry Reconstruction from Implicit Neural Representations

  2024cites this paper
• HOLI-1-to-3: Transient-Enhanced Holistic Image-to-3D Generation

  2024cites this paper
• Multi-view stereo algorithms based on deep learning: a survey

  2024cites this paper
• Pre-training Sequence, Structure, and Surface Features for Comprehensive Protein Representation Learning

  2024cites this paper
• A High-Performance Learning-Based Framework for Monocular 3-D Point Cloud Reconstruction

  2024cites this paper
• PIDSNeRF: pose interpolation depth supervision neural radiance fields for view synthesis from challenging input

  2024cites this paper
• Three-Dimensional Object Reconstruction from Multi-View Images with Disentangled Attribute Flow

  2024cites this paper
• OSN: Infinite Representations of Dynamic 3D Scenes from Monocular Videos

  2024cites this paper
• Generation of simple motion sequences for supporting manual operation of rotary cranes

  2024cites this paper
• SVDTree: Semantic Voxel Diffusion for Single Image Tree Reconstruction

  2024cites this paper
• Transfer Learning-based Approach for 3D Reconstruction from a Single 2D Image

  2024cites this paper
• Digitalization of 3-D Human Bodies: A Survey

  2024cites this paper
• Transfer Learning-based Approach for 3D Reconstruction from a Single 2D Image

  2024cites this paper
• TreeSBA: Tree-Transformer for Self-Supervised Sequential Brick Assembly

  2024cites this paper
• Underground Mapping and Localization Based on Ground-Penetrating Radar

  2024cites this paper
• Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation

  2023cites this paper
• Neural Vector Fields: Implicit Representation by Explicit Learning

  2023cites this paper
• Replay Attention and Data Augmentation Network for 3-D Face and Object Reconstruction

  2023cites this paper
• Shape Morphing as a Minimal Path in the Graph of Cubified Shapes

  2023cites this paper
• CPG3D: Cross-Modal Priors Guided 3D Object Reconstruction

  2023cites this paper
• A Review of Deep Learning-Powered Mesh Reconstruction Methods

  2023cites this paper
• NeRF-DS: Neural Radiance Fields for Dynamic Specular Objects

  2023cites this paper
• NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusion

  2023cites this paper
• Visibility-Aware Pixelwise View Selection for Multi-View Stereo Matching

  2023cites this paper
• Active velocity estimation using light curtains via self-supervised multi-armed bandits

  2023influential citation
• SegForestNet: Spatial-Partitioning-Based Aerial Image Segmentation

  2023cites this paper
• Deep learning-based 3D reconstruction: a survey

  2023influential citation