Adaptive Sampling Towards Fast Graph Representation Learning

Graph Convolutional Networks (GCNs) have become a crucial tool on learning representations of graph vertices. The main challenge of adapting GCNs on large-scale graphs is the scalability issue that it incurs heavy cost both in computation and memory due to the uncontrollable neighborhood expansion across layers. In this paper, we accelerate the training of GCNs through developing an adaptive layer-wise sampling method. By constructing the network layer by layer in a top-down passway, we sample the lower layer conditioned on the top one, where the sampled neighborhoods are shared by different parent nodes and the over expansion is avoided owing to the fixed-size sampling. More importantly, the proposed sampler is adaptive and applicable for explicit variance reduction, which in turn enhances the training of our method. Furthermore, we propose a novel and economical approach to promote the message passing over distant nodes by applying skip connections. Intensive experiments on several benchmarks verify the effectiveness of our method regarding the classification accuracy while enjoying faster convergence speed.

• Publication year

  2018

• Venue

  Neural Information Processing Systems

• Publication date

  2018-09-01

• Fields of study

  Computer Science

• Identifiers
  arXiv 1809.05343
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling

  2018influential reference
• Graph Attention Networks

  2017cited by this paper
• Stochastic Training of Graph Convolutional Networks with Variance Reduction

  2017cited by this paper
• Inductive Representation Learning on Large Graphs

  2017influential reference
• Robust Spatial Filtering With Graph Convolutional Neural Networks

  2017cited by this paper
• Protein Interface Prediction using Graph Convolutional Networks

  2017cited by this paper
• Attention is All you Need

  2017cited by this paper
• node2vec: Scalable Feature Learning for Networks

  2016cited by this paper
• Learning Convolutional Neural Networks for Graphs

  2016cited by this paper
• Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

  2016cited by this paper
• Semi-Supervised Classification with Graph Convolutional Networks

  2016influential reference
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

  2016cited by this paper
• TensorFlow: A system for large-scale machine learning

  2016cited by this paper
• Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs

  2016cited by this paper
• Convolutional Networks on Graphs for Learning Molecular Fingerprints

  2015cited by this paper
• Deep Residual Learning for Image Recognition

  2015influential reference
• Diffusion-Convolutional Neural Networks

  2015influential reference
• Deep Convolutional Networks on Graph-Structured Data

  2015cited by this paper
• LINE: Large-scale Information Network Embedding

  2015cited by this paper
• DeepWalk: online learning of social representations

  2014cited by this paper
• Spectral Networks and Locally Connected Networks on Graphs

  2013cited by this paper
• Robust and Scalable Graph-Based Semisupervised Learning

  2012cited by this paper
• Collective Classi!cation in Network Data

  2008cited by this paper
• Collective Classification in Network Data

  2008influential reference
• An Experimental Investigation of Graph Kernels on a Collaborative Recommendation Task

  2006influential reference

• Optimization-Free Graph Embedding via Distributional Kernel for Community Detection

  2026cites this paper
• UniTrain: A universal iterative semi-supervised training framework for graph representation learning.

  2026cites this paper
• InfGraND: An Influence-Guided GNN-to-MLP Knowledge Distillation

  2026cites this paper
• A survey on data management for Out-of-Core GNN systems

  2026cites this paper
• AutoGNN: End-to-End Hardware-Driven Graph Preprocessing for Enhanced GNN Performance

  2026cites this paper
• Graph Negative Feedback Bias Correction Framework for Adaptive Heterophily Modeling

  2026cites this paper
• MQ-GNN: A Multi-Queue Pipelined Architecture for Scalable and Efficient GNN Training

  2026influential citation
• Label-frugal satellite image change detection with generative virtual exemplar learning

  2025cites this paper
• Which is playing a key role on sampling-based large-scale GNNs, sampling or iteration?

  2025cites this paper
• Dynamic Link Prediction Analysis Using Temporal Adaptive Graph Convolutional Long Short-Term Memory

  2025cites this paper
• FaaSGNN: Enabling Memory Efficient and Low Latency GNN Inference Services with Serverless Computing

  2025cites this paper
• Image augmentation with invertible networks in interactive satellite image change detection

  2025cites this paper
• BLISS: Bandit Layer Importance Sampling Strategy for Efficient Training of Graph Neural Networks

  2025cites this paper
• Multi-grained contrastive-learning driven MLPs for node classification

  2025cites this paper
• Simultaneous tracking of objects with loose context constraints from multiple views: human–human interaction paradigm

  2025cites this paper
• Feature points classification of computerized numerical control finishing tool path based on graph neural network

  2025cites this paper
• A universal sampling method based on feature and structural comprehensive proximity measure

  2025cites this paper
• RingSampler: GNN sampling on large-scale graphs with io_uring

  2025cites this paper
• Accurate and Scalable Graph Neural Networks via Message Invariance

  2025cites this paper
• PyG 2.0: Scalable Learning on Real World Graphs

  2025cites this paper
• Hierarchical graph sampling based minibatch learning with chain preservation and variance reduction

  2025cites this paper
• GNNBoost: Accelerating sampling-based GNN training on large scale graph by optimizing data preparation

  2025cites this paper
• Optimization of Graph Neural Networks Training Using Graph Reordering

  2025cites this paper
• CrossETR: A Semantic-Driven Framework for Entity Matching Across Images and Graph

  2025cites this paper
• Enhanced multi objective graph learning approach for optimizing traffic speed prediction on spatial and temporal features

  2025cites this paper
• HGP-IC: graph neural networks via hotness-based partitioning and information compensation

  2025cites this paper
• Geometric and topological structure-induced large-scale graph learning for social and information networks

  2025cites this paper
• Democratic Recommendation with User and Item Representatives Produced by Graph Condensation

  2025cites this paper
• HFLR: Optimizing GNN Training via High-Fixed-Low-Resampling

  2025cites this paper
• A Comprehensive Analysis of Graph Neural Networks Training at Different Scales

  2025cites this paper
• A universal strategy for smoothing deceleration in deep graph neural networks

  2025cites this paper
• Theoretical Learning Performance of Graph Neural Networks: The Impact of Jumping Connections and Layer-wise Sparsification

  2025cites this paper
• Neighbor-Sampling Based Momentum Stochastic Methods for Training Graph Neural Networks

  2025cites this paper
• FS-GNN: Improving Fairness in Graph Neural Networks via Joint Sparsification

  2025cites this paper
• An Embedding Algorithm Based on Neighborhood Aggregation for Attribute Graphs

  2025cites this paper
• Capsule: An Out-of-Core Training Mechanism for Colossal GNNs

  2025cites this paper
• Dual-View Desynchronization Hypergraph Learning for Dynamic Hyperedge Prediction

  2025cites this paper
• Hybrid graph convolutional and deep convolutional networks for enhanced pavement crack detection

  2025cites this paper
• Graph classification by converting static graphs into dynamic evolutionary sequences

  2025cites this paper
• DCI: A Coordinated Allocation and Filling Workload-Aware Dual-Cache Allocation GNN Inference Acceleration System

  2025cites this paper
• Graph Neural Network (GNN) and its Application: A State-of-the-Art Survey

  2025cites this paper
• Comprehensive Review of Graph Neural Networks: Challenges, Classification, Architectures, Applications, and Potential Utility in Bioinformatics

  2025cites this paper
• Lightweight Embeddings with Graph Rewiring for Collaborative Filtering

  2025cites this paper
• Spatial-temporal context-aware network for 3D-Craft generation

  2025cites this paper
• A construction of hyperbolic tree-based graph convolutional networks utilizing the Padé approximation

  2025cites this paper
• Staleness-Based Subgraph Sampling for Training GNNs on Large-Scale Graphs

  2025cites this paper
• Community Detection Based on Graph Neural Network

  2025cites this paper
• HG-PAD: Heterogeneous Graph Structure Learning Aided Performance Anomaly Diagnosis in Microservice Systems

  2025cites this paper
• TTGL: Large-scale Multi-scenario Universal Graph Learning at TikTok

  2025cites this paper
• On the Interplay between Graph Structure and Learning Algorithms in Graph Neural Networks

  2025cites this paper
• HopGAT: A multi-hop graph attention network with heterophily and degree awareness

  2025cites this paper
• Effectiveness and efficiency: label-aware hierarchical subgraph learning for protein-protein interaction

  2024cites this paper
• Unifews: You Need Fewer Operations for Efficient Graph Neural Networks

  2024cites this paper
• Revisiting Edge Perturbation for Graph Neural Network in Graph Data Augmentation and Attack

  2024cites this paper
• Scalable Graph Self-Supervised Learning

  2024cites this paper
• Incorporating Multi-Level Sampling with Adaptive Aggregation for Inductive Knowledge Graph Completion

  2024cites this paper
• Hierarchical Multi-Relational Graph Representation Learning for Large-Scale Prediction of Drug-Drug Interactions

  2024cites this paper
• TASER: Temporal Adaptive Sampling for Fast and Accurate Dynamic Graph Representation Learning

  2024cites this paper
• Scalable and Efficient Temporal Graph Representation Learning via Forward Recent Sampling

  2024cites this paper
• SHINE: A Scalable Heterogeneous Inductive Graph Neural Network for Large Imbalanced Datasets

  2024cites this paper
• FMLGLN: Fast Multi-layer Graph Linear Network

  2024cites this paper
• XGNN: Boosting Multi-GPU GNN Training via Global GNN Memory Store

  2024cites this paper
• Graph Regularized Encoder Training for Extreme Classification

  2024cites this paper
• Acceleration Algorithms in GNNs: A Survey

  2024cites this paper
• FairSample: Training Fair and Accurate Graph Convolutional Neural Networks Efficiently

  2024cites this paper
• Hierarchical spiking neural P systems with weights on multiple channels for graph-based node classification

  2024cites this paper
• Graph Coarsening via Supervised Granular-Ball for Scalable Graph Neural Network Training

  2024cites this paper
• Scaling New Heights: Transformative Cross-GPU Sampling for Training Billion-Edge Graphs

  2024cites this paper
• GERT: Graph Encoder Representations from Transformers

  2024cites this paper
• PromptGCN: Bridging Subgraph Gaps in Lightweight GCNs

  2024cites this paper
• Haste Makes Waste: A Simple Approach for Scaling Graph Neural Networks

  2024cites this paper
• HeteroSample: Meta-Path Guided Sampling for Heterogeneous Graph Representation Learning

  2024cites this paper
• DropNaE: Alleviating irregularity for large-scale graph representation learning

  2024cites this paper
• Sub-sampling graph neural networks for genomic prediction of quantitative phenotypes

  2024cites this paper
• DiffusionE: Reasoning on Knowledge Graphs via Diffusion-based Graph Neural Networks

  2024cites this paper
• Survey on Graph Neural Networks

  2024cites this paper
• Finding core labels for maximizing generalization of graph neural networks

  2024cites this paper
• On Pipelined GCN with Communication-Efficient Sampling and Inclusion-Aware Caching

  2024cites this paper
• A Multi-Graph Fusion Framework for Patient Representation Learning

  2024cites this paper
• FastGL: A GPU-Efficient Framework for Accelerating Sampling-Based GNN Training at Large Scale

  2024influential citation
• CDFGNN: a Systematic Design of Cache-based Distributed Full-Batch Graph Neural Network Training with Communication Reduction

  2024cites this paper
• Graph Batch Coarsening framework for scalable graph neural networks

  2024cites this paper
• Gradient Scarcity in Graph Learning with Bilevel Optimization

  2024cites this paper
• Towards Scalable and Deep Graph Neural Networks via Noise Masking

  2024cites this paper
• OAK: Enriching Document Representations using Auxiliary Knowledge for Extreme Classification

  2024cites this paper
• E2GCL: Efficient and Expressive Contrastive Learning on Graph Neural Networks

  2024cites this paper
• Scalable and Effective Implicit Graph Neural Networks on Large Graphs

  2024influential citation
• Sylvie: 3D-Adaptive and Universal System for Large-Scale Graph Neural Network Training

  2024cites this paper
• Explainable Graph Neural Networks: An Application to Open Statistics Knowledge Graphs for Estimating House Prices

  2024cites this paper
• DropEdge not Foolproof: Effective Augmentation Method for Signed Graph Neural Networks

  2024cites this paper
• Graph Contrastive Multi-view Learning: A Pre-training Framework for Graph Classification

  2024cites this paper
• In situ neighborhood sampling for large-scale GNN training

  2024cites this paper
• Centrality-based and similarity-based neighborhood extension in graph neural networks

  2024cites this paper
• FastHGNN: A New Sampling Technique for Learning with Hypergraph Neural Networks

  2024cites this paper
• DiskGNN: Bridging I/O Efficiency and Model Accuracy for Out-of-Core GNN Training

  2024cites this paper
• NPA: Improving Large-scale Graph Neural Networks with Non-parametric Attention

  2024cites this paper
• DGNN-MN: Dynamic Graph Neural Network via memory regenerate and neighbor propagation

  2024cites this paper
• Adaptive unified contrastive learning with graph-based feature aggregator for imbalanced medical image classification

  2024cites this paper
• Extracting High-Fidelity Smaller Scale Subgraphs of Complex Networks by Edge-Reinforced Random Walk

  2024cites this paper
• Graph Convolutional Network For Semi-supervised Node Classification With Subgraph Sketching

  2024cites this paper