Communication-Efficient Distributed Statistical Inference

ABSTRACT We present a communication-efficient surrogate likelihood (CSL) framework for solving distributed statistical inference problems. CSL provides a communication-efficient surrogate to the global likelihood that can be used for low-dimensional estimation, high-dimensional regularized estimation, and Bayesian inference. For low-dimensional estimation, CSL provably improves upon naive averaging schemes and facilitates the construction of confidence intervals. For high-dimensional regularized estimation, CSL leads to a minimax-optimal estimator with controlled communication cost. For Bayesian inference, CSL can be used to form a communication-efficient quasi-posterior distribution that converges to the true posterior. This quasi-posterior procedure significantly improves the computational efficiency of Markov chain Monte Carlo (MCMC) algorithms even in a nondistributed setting. We present both theoretical analysis and experiments to explore the properties of the CSL approximation. Supplementary materials for this article are available online.

• Publication year

  2016

• Venue

  Journal of the American Statistical Association

• Publication date

  2016-05-25

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1080/01621459.2018.1429274
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Stochastic Variance Reduced Gradient Methods by Sampling Extra Data with Replacement

  2017cited by this paper
• Efficient Distributed Learning with Sparsity

  2016cited by this paper
• Variational Consensus Monte Carlo

  2015cited by this paper
• Communication-Efficient Distributed Optimization of Self-Concordant Empirical Loss

  2015cited by this paper
• DiSCO: Distributed Optimization for Self-Concordant Empirical Loss

  2015cited by this paper
• Asynchronous Gibbs Sampling

  2015cited by this paper
• Distributed Stochastic Variance Reduced Gradient Methods and A Lower Bound for Communication Complexity

  2015influential reference
• Communication-efficient sparse regression: a one-shot approach

  2015influential reference
• Communication lower bounds for statistical estimation problems via a distributed data processing inequality

  2015influential reference
• Deep learning with Elastic Averaging SGD

  2014cited by this paper
• Theory of Convex Optimization for Machine Learning

  2014cited by this paper
• Introductory Lectures on Convex Optimization - A Basic Course

  2014cited by this paper
• Divide and recombine (D&R): Data science for large complex data

  2014cited by this paper
• Firefly Monte Carlo: Exact MCMC with Subsets of Data

  2014cited by this paper
• Iterative Hessian Sketch: Fast and Accurate Solution Approximation for Constrained Least-Squares

  2014cited by this paper
• Optimality guarantees for distributed statistical estimation

  2014cited by this paper
• On Communication Cost of Distributed Statistical Estimation and Dimensionality

  2014cited by this paper
• A general framework for updating belief distributions

  2013cited by this paper
• Asymptotically Exact, Embarrassingly Parallel MCMC

  2013influential reference
• Principal Component Analysis and Higher Correlations for Distributed Data

  2013cited by this paper
• Accelerating Stochastic Gradient Descent using Predictive Variance Reduction

  2013cited by this paper
• On asymptotically optimal confidence regions and tests for high-dimensional models

  2013influential reference
• Communication-Efficient Distributed Optimization using an Approximate Newton-type Method

  2013cited by this paper
• Parallelizing MCMC via Weierstrass Sampler

  2013influential reference
• Communication-efficient algorithms for statistical optimization

  2012influential reference
• Distributed matrix completion and robust factorization

  2011cited by this paper
• A scalable bootstrap for massive data

  2011cited by this paper
• Understanding GPU Programming for Statistical Computation: Studies in Massively Parallel Massive Mixtures

  2010cited by this paper
• Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling

  2010cited by this paper
• Restricted Eigenvalue Properties for Correlated Gaussian Designs

  2010cited by this paper
• A unified framework for high-dimensional analysis of $M$-estimators with decomposable regularizers

  2009cited by this paper
• Communication-optimal Parallel and Sequential QR and LU Factorizations

  2008cited by this paper
• Communication-optimal parallel and sequential QR and LU factorizations: theory and practice

  2008cited by this paper
• An MCMC approach to classical estimation

  2003cited by this paper

• Accelerated Methods with Complexity Separation Under Data Similarity for Federated Learning Problems

  2026cites this paper
• Distributed robust estimation and inference with contaminated data

  2026cites this paper
• Distributed Convoluted Rank Regression for Non-Shareable Data under Non-Additive Losses

  2026influential citation
• Statistical Inference for High Dimensional Regression with Proxy Data

  2026cites this paper
• Communication-efficient distributed composite quantile regression via convolution smoothing and poisson subsampling

  2026cites this paper
• Large-scale quantile regression with covariates missing at random

  2026cites this paper
• Robust Distributed High-Dimensional Regression: A Convoluted Rank Approach

  2026cites this paper
• A likelihood approach to proper analysis of secondary outcomes in matched case-control studies

  2026cites this paper
• An adaptive-to-model distributed hybrid test for conditional independence

  2026cites this paper
• A Communication-Efficient Distributed Algorithm for Online Monitoring in Big Data Systems

  2026cites this paper
• Communication-efficient and distributed-oracle estimation for high-dimensional quantile regression

  2026cites this paper
• Emerging Knowledge Trend in Statistical Research: A Content-Based Analysis Using Covariate-Assisted Dynamic Topic Model

  2026cites this paper
• Distributed stochastic optimization algorithm based on Markov sampling

  2026cites this paper
• A Communication Efficient Boosting Method for Distributed Spectral Clustering

  2026cites this paper
• Enhancements of communication-efficient distributed statistical inference and its privacy preservation

  2026cites this paper
• A Review of Recent Advances in High‐Dimensional Quantile Regression

  2026cites this paper
• One-Shot Federated Ridge Regression: Exact Recovery via Sufficient Statistic Aggregation

  2026cites this paper
• Few-Round Distributed Principal Component Analysis: Closing the Statistical Efficiency Gap by Consensus

  2025cites this paper
• Distributed Mallows Model Averaging for Ridge Regressions

  2025cites this paper
• Distributed algorithm for best subset regression

  2025cites this paper
• Bayesian Federated Inference with Systematically Missing Features

  2025cites this paper
• Distributed optimization for penalized regression in massive compositional data

  2025cites this paper
• Robust and efficient sparse learning over networks: a decentralized surrogate composite quantile regression approach

  2025cites this paper
• Exploring Temporal Similarity for Joint Computation and Communication in Online Distributed Optimization

  2025cites this paper
• Efficient distributed estimation for expectile regression in increasing dimensions

  2025cites this paper
• Distributed Feature Screening via Inverse Regression

  2025cites this paper
• A Selective Overview of Quantile Regression for Large-Scale Data

  2025cites this paper
• On Robust Aggregation for Distributed Data

  2025cites this paper
• Communication-Efficient Distributed Sparse Learning with Oracle Property and Geometric Convergence

  2025influential citation
• Ensemble of Sequential Learning Models With Distributed Data Centers and Its Applications

  2025cites this paper
• Constrained Over-the-Air Model Updating for Wireless Online Federated Learning with Delayed Information

  2025cites this paper
• Quadratic discriminant analysis in distributed frameworks

  2025cites this paper
• A Communication-Efficient Distributed Algorithm for Learning with Heterogeneous and Structurally Incomplete Multi-Site Data

  2025cites this paper
• Scalable and Communication-Efficient Varying Coefficient Mixed Effect Models: Methodology, Theory, and Applications

  2025cites this paper
• Communication-Efficient Distributed Estimation and Computation Using Skew-Normal Distribution

  2025cites this paper
• Distributed learning for high-dimensional factor augmented regression models

  2025cites this paper
• Privacy-Enhancing Sequential Learning under Heterogeneous Selection Bias in Multi-Site EHR Data

  2025cites this paper
• Unified Distributed Estimation Framework for Sufficient Dimension Reduction Based on Conditional Moments

  2025cites this paper
• Communication-efficient distributed robust variable selection for heterogeneous massive data

  2025cites this paper
• Federated Learning of Quantile Inference under Local Differential Privacy

  2025cites this paper
• Efficient Quantization Mean Estimation for Distributed Learning*

  2025cites this paper
• Communication-efficient estimation and inference for high-dimensional quantile regression with unmeasured confounders

  2025influential citation
• Distributed Semi‐Supervised Single‐Index Model With Corruption

  2025cites this paper
• Adaptive Decentralized Federated Learning for Robust Optimization

  2025cites this paper
• Renewable Estimation and Heterogeneity Detection Under Heterogeneous Estimating Equation Settings

  2025cites this paper
• Distributed quadratic interpolation estimation for large-scale quantile regression

  2025cites this paper
• Scalable and Distributed Individualized Treatment Rules for Massive Datasets

  2025cites this paper
• Communication‐Efficient Feature Screening for Ultrahigh‐Dimensional Data Under Quantile Regression

  2025cites this paper
• A distributed algorithm for fuzzy support vector machine on multi-source data-driven credit scoring

  2025cites this paper
• A novel communication-efficient heterogeneous federated positive and unlabeled learning method for credit scoring

  2025cites this paper
• Efficient Semiparametric Inference for Distributed Data with Blockwise Missingness

  2025cites this paper
• Distributed Estimation for ℓ0-Constrained Quantile Regression Using Iterative Hard Thresholding

  2025cites this paper
• Communication-efficient model averaging prediction for massive data with asymptotic optimality

  2025cites this paper
• Communication-efficient federated learning of temporal effects on opioid use disorder with data from distributed research networks

  2025cites this paper
• High‐Performance Statistical Computing (HPSC): Challenges, Opportunities, and Future Directions

  2025cites this paper
• Adaptive Divide and Conquer with Two Rounds of Communication

  2025cites this paper
• Adaptive weighted approach for high-dimensional statistical learning and inference

  2025cites this paper
• Communication-efficient estimation and inference for high-dimensional longitudinal data

  2025cites this paper
• Privacy-preserving estimation for non-randomly distributed data

  2025cites this paper
• Late Fusion Multi-task Learning for Semiparametric Inference with Nuisance Parameters

  2025cites this paper
• Distributed Estimation and Algorithm for Distributed Outcome Dependent Subsampling in Generalized Linear Regression with Large-scale Data

  2025cites this paper
• Bayesian Federated Inference for regression models based on non-shared medical center data

  2025cites this paper
• Distributed Estimation and Inference for High-Dimensional Confounded Models

  2025cites this paper
• Privacy-preserving communication-efficient spectral clustering for distributed multiple networks

  2025cites this paper
• Development of time to event prediction models using federated learning

  2025cites this paper
• D3MI: an efficient and powerful federated imputation method for bias reduction in the analysis of distributed incomplete data by accounting for within-site correlation and between-site heterogeneity

  2025influential citation
• Robust Distributed Estimation for Modal Regression under Least Squares Approximation

  2025cites this paper
• Communication-efficient transfer learning for adaptive Huber regression

  2025cites this paper
• Communication-Efficient Pilot Estimation for Non-Randomly Distributed Data in Diverging Dimensions

  2025cites this paper
• Distributed iterative hard thresholding for variable selection in Tobit models

  2025cites this paper
• DisC2o-HD: Distributed causal inference with covariates shift for analyzing real-world high-dimensional data

  2025cites this paper
• Unlocking efficiency in real-world collaborative studies: a multi-site international study with one-shot lossless GLMM algorithm

  2025cites this paper
• Regularized Adaptive Huber Matrix Regression and Distributed Learning

  2025cites this paper
• Distributed Nyström Approximation with Convex Lipschitz Loss

  2025cites this paper
• Dimension Reduction for Large-Scale Federated Data: Statistical Rate and Asymptotic Inference

  2025cites this paper
• Divide-and-conquer offline policy evaluation for contextual bandits

  2025cites this paper
• Statistical Theory of Multi-stage Newton Iteration Algorithm for Online Continual Learning

  2025influential citation
• Distributed Semi-Supervised Inference for Generalized Linear Models With Block-Wise Missing Covariates

  2025cites this paper
• Robust distributed precision matrix estimation for high-dimensional data

  2025cites this paper
• Learning Centre Partitions from Summaries

  2025cites this paper
• Linear Transformers Implicitly Discover Unified Numerical Algorithms

  2025cites this paper
• Heterogeneity-aware debiased machine learning for high-dimensional partially linear models

  2025cites this paper
• A locally sequentially reweighted gradient descent estimator to enhance statistical efficiency for decentralized federated learning

  2025cites this paper
• Robust Clustered Federated Learning for Heterogeneous High-dimensional Data

  2025cites this paper
• Bias-Corrected Method of Moments Estimation of the Hurst Parameter for Improved Option Pricing Under the Fractional Black-Scholes Model

  2025cites this paper
• Collaborative Inference for Accelerated Failure Time Model Using Clinical Center‐Level Summary Statistics

  2025cites this paper
• Decentralized Quantile Regression for Feature-Distributed Massive Datasets with Privacy Guarantees

  2025cites this paper
• Collaborative Inference for Sparse High-Dimensional Models with Non-Shared Data

  2025influential citation
• Communication-Efficient l0 Penalized Least Square

  2025influential citation
• Federated double machine learning for high-dimensional semiparametric models.

  2025cites this paper
• Distributed Composite Quantile Regression for High-Dimensional Data

  2025cites this paper
• Integrating high-dimensional censored data under privacy constraints via localized computations

  2025cites this paper
• Robust Variational Bayes by Min-Max Median Aggregation

  2025cites this paper
• Summary of Reservoir Model Parameter Inversion Technology based on Artificial Intelligence

  2025cites this paper
• A critical assessment of matching-adjusted indirect comparisons in relation to target populations

  2025cites this paper
• An algorithm for distributed parameter estimation in modal regression models

  2025cites this paper
• Distributed Quasi-Newton Algorithm for Non-Randomly Stored Data

  2025cites this paper
• COLA-GLM: collaborative one-shot and lossless algorithms of generalized linear models for decentralized observational healthcare data

  2025cites this paper
• Optimal distributed Poisson subsampling for modal regression with massive data

  2025influential citation
• Sequential Data Integration Under Dataset Shift

  2024cites this paper