Large Scale Distributed Deep Networks

No abstract is available for this paper.

• Publication year

  2012

• Venue

  Neural Information Processing Systems

• Publication date

  2012-12-03

• Fields of study

  Computer Science

• Identifiers

  No identifiers available.

• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Distributed GraphLab: A Framework for Machine Learning in the Cloud

  2012cited by this paper
• Distributed GraphLab: A Framework for Machine Learning in the Cloud

  2012cited by this paper
• Efficient BackProp

  2012cited by this paper
• Scalable stacking and learning for building deep architectures

  2012cited by this paper
• Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition

  2012influential reference
• Deep Neural Networks for Acoustic Modeling in Speech Recognition

  2012influential reference
• Multi-column deep neural networks for image classification

  2012cited by this paper
• An Analysis of Single-Layer Networks in Unsupervised Feature Learning

  2011cited by this paper
• Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

  2011cited by this paper
• Distributed delayed stochastic optimization

  2011cited by this paper
• Building high-level features using large scale unsupervised learning

  2011cited by this paper
• Improving the speed of neural networks on CPUs

  2011cited by this paper
• A reliable effective terascale linear learning system

  2011cited by this paper
• On optimization methods for deep learning

  2011cited by this paper
• Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent

  2011cited by this paper
• Parallelized Stochastic Gradient Descent

  2010cited by this paper
• Deep, Big, Simple Neural Nets for Handwritten Digit Recognition

  2010cited by this paper
• Distributed Training Strategies for the Structured Perceptron

  2010cited by this paper
• Deep learning via Hessian-free optimization

  2010cited by this paper
• Large-scale deep unsupervised learning using graphics processors

  2009influential reference
• Efficient Large-Scale Distributed Training of Conditional Maximum Entropy Models

  2009cited by this paper
• Hash Kernels

  2009cited by this paper
• Slow Learners are Fast

  2009cited by this paper
• Learning Multiple Layers of Features from Tiny Images

  2009cited by this paper
• ImageNet: A large-scale hierarchical image database

  2009cited by this paper
• A unified architecture for natural language processing: deep neural networks with multitask learning

  2008cited by this paper
• MapReduce: simplified data processing on large clusters

  2008influential reference
• A scalable modular convex solver for regularized risk minimization

  2007cited by this paper
• A Neural Probabilistic Language Model

  2003cited by this paper
• Stochastic Gradient Learning in Neural Networks

  1991cited by this paper

• Self-Evolving Distributed Memory Architecture for Scalable AI Systems

  2026cites this paper
• Deep Learning for Image Watermarking: A Comprehensive Review and Analysis of Techniques, Challenges, and Applications

  2026cites this paper
• Asynchronous Heavy-Tailed Optimization

  2026cites this paper
• PRISM: Profiling-Free Symbolic Memory-Driven Strategy Planner for Large DNN Model Training

  2026cites this paper
• DHP: Efficient Scaling of MLLM Training with Dynamic Hybrid Parallelism

  2026cites this paper
• A robust federated aggregation algorithm for multimodal data in smart grid scenarios

  2026cites this paper
• ORBITFLOW: SLO-Aware Long-Context LLM Serving with Fine-Grained KV Cache Reconfiguration

  2026cites this paper
• Make Every Draft Count: Hidden State based Speculative Decoding

  2026cites this paper
• Layer-Parallel Training for Transformers

  2026cites this paper
• SIM: Accelerating Distributed DNN Training by Exploring Gradient Similarity

  2026cites this paper
• Why Inference in Large Models Becomes Decomposable After Training

  2026influential citation
• Age of Sensing Empowered Holographic ISAC Framework for nextG Wireless Networks: A VAE and DRL Approach

  2026cites this paper
• Communication-Efficient and Privacy-Adaptable Mechanism - a Federated Learning Scheme with Convergence Analysis

  2026cites this paper
• Distributed Large Models Training Optimization With Real-Time Wireless Channel Feedback

  2026cites this paper
• Provably Convergent Decentralized Optimization over Directed Graphs under Generalized Smoothness

  2026cites this paper
• Neural Network Training Acceleration Based on Hybrid Data and Model Parallelism

  2026cites this paper
• Parallel Track Transformers: Enabling Fast GPU Inference with Reduced Synchronization

  2026cites this paper
• Sassy: SmartNIC-Assisted Notification Delivery for μs-Scale RDMA Workloads

  2026cites this paper
• Unpacking distributed machine learning: A unified taxonomy, formal foundations, and the rise of emerging paradigms

  2026cites this paper
• Distributed Linearly Separable Computation with Arbitrary Heterogeneous Data Assignment

  2026cites this paper
• Mitigating Server-Side Communication Bottlenecks in Distributed Learning With Round-Robin Participant Coordination

  2026cites this paper
• Advancing RDMA Scalability With High Performance

  2026cites this paper
• Tight Analysis of Decentralized SGD: A Markov Chain Perspective

  2026cites this paper
• Ordered Local Momentum for Asynchronous Distributed Learning under Arbitrary Delays

  2026cites this paper
• Revisiting Parameter Server in LLM Post-Training

  2026cites this paper
• Do Neural Networks Lose Plasticity in a Gradually Changing World?

  2026cites this paper
• Perception of AI-assisted health communication via ChatGPT in Hungary

  2026cites this paper
• Enhancing Remote Direct Memory Access Network With Loss-Tolerant Flow Control for Distributed AI Clusters

  2026cites this paper
• Enhancing Diabetes Prediction in Health Informatics Through Software analysis and Machine Learning Synergy

  2025cites this paper
• Advanced Methods for Optimization of Neural Networks for Signal Processing: A Survey

  2025cites this paper
• A Dynamics Theory of RMSProp-Based Implicit Regularization in Deep Low-Rank Matrix Factorization

  2025cites this paper
• Hardware Acceleration of Scalable Artificial Neural Network Inference on FPGA

  2025cites this paper
• Adversarial Robustness of Nonparametric Regression

  2025cites this paper
• Cross-platform Fake Review Detection: A Comparative Analysis of Supervised and Deep Learning Models

  2025cites this paper
• PORSCHE: Progressive Optimization and Robust Spatial Convolution for Hybrid Enhancement in Visible-Infrared Vehicle Re-Identification

  2025cites this paper
• Artificial Intelligence in Cyber Defense: A Study of Modern Security Solutions

  2025cites this paper
• Introducing a Quality-Driven Approach for Federated Learning

  2025cites this paper
• Batches Stabilize the Minimum Norm Risk in High-Dimensional Overparametrized Linear Regression

  2025cites this paper
• Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients

  2025cites this paper
• Straggler Mitigation in Distributed Deep Learning: A Cluster-Based Hybrid Synchronization Approach

  2025cites this paper
• Database management systems for artificial intelligence: Comparative analysis of postgre SQL and MongoDB

  2025cites this paper
• A Comparative Study of Feature Extraction and Classification Techniques for Punjabi Speech Recognition

  2025cites this paper
• HarMoEny: Efficient Multi-GPU Inference of MoE Models

  2025cites this paper
• An Adaptive and Scalable Framework for Resource-Efficient Deployment of Mixture of Experts in LLM-Based Intelligent IoT Networks

  2025cites this paper
• OrchMLLM: Orchestrate Multimodal Data with Batch Post-Balancing to Accelerate Multimodal Large Language Model Training

  2025cites this paper
• Pipeline parallelism with reduced network communication for efficient compute-intensive neural network training

  2025cites this paper
• RDQKL: Robust Decentralized Quantum Kernel Learning for Noisy and Adversarial Environment

  2025cites this paper
• Mitigating Scalability Walls of RDMA-based Container Networks

  2025cites this paper
• SecureFedPROM: A Zero-Trust Federated Learning Approach With Multi-Criteria Client Selection

  2025cites this paper
• Towards Trustworthy Federated Learning with Untrusted Participants

  2025cites this paper
• Understanding Stragglers in Large Model Training Using What-if Analysis

  2025cites this paper
• The Adaptive Complexity of Finding a Stationary Point

  2025cites this paper
• Performance Characterization of Distributed Deep Learning Strategies: A Quantitative Evaluation of DDP, FSDP, and Parameter Server Architectures on GPU Clusters

  2025cites this paper
• Layer-wise Quantization for Quantized Optimistic Dual Averaging

  2025influential citation
• Enhancing Efficiency and Reducing the Carbon Footprint of Cloud-Based Healthcare Applications through Optimal Data Preprocessing

  2025cites this paper
• Bigger is not always better: The importance of human-scale language modeling for psycholinguistics

  2025cites this paper
• Integration of genomic and clinical data for predicting psoriasis onset

  2025cites this paper
• Self-Healing Infrastructure: Leveraging Reinforcement Learning for Autonomous Cloud Recovery and Enhanced Resilience

  2025cites this paper
• ByzFL: Research Framework for Robust Federated Learning

  2025cites this paper
• QKV Projections Require a Fraction of Their Memory

  2025cites this paper
• Efficient Hardware Scaling and Diminishing Returns in Large-Scale Training of Language Models

  2025cites this paper
• RLFreeze: Accelerating Distributed Freezing Training with Reinforcement Learning

  2025cites this paper
• ATTENTION2D: Communication Efficient Distributed Self-Attention Mechanism

  2025cites this paper
• GC-Fed: Gradient Centralized Federated Learning with Partial Client Participation

  2025cites this paper
• Software-defined cloud–optical networks for long-haul geographically distributed machine learning

  2025cites this paper
• Scalable and Load-Balanced Full-Graph GNN Training on Multiple GPUs

  2025cites this paper
• Waste Not, Want Not; Recycled Gumbel Noise Improves Consistency in Natural Language Generation

  2025cites this paper
• ADNeuroNet: a neuroevolution-based neural network algorithm for the diagnosis of neurodegenerative diseases

  2025cites this paper
• Communication-Efficient Vertical Federated Learning via Compressed Error Feedback

  2025cites this paper
• CGFL: A Robust Federated Learning Approach for Intrusion Detection Systems Based on Data Generation

  2025cites this paper
• Hierarchical Gradient Coding: From Optimal Design to Privacy at Intermediate Nodes

  2025cites this paper
• Enhancing Collaborative Deep Learning with Swarm Intelligence and Federated Optimization

  2025cites this paper
• Routing for Large ML Models

  2025cites this paper
• Federated Learning-Based Framework to Improve the Operational Efficiency of an Articulated Robot Manufacturing Environment

  2025cites this paper
• Parallelization of Recurrent Neural Network training algorithm with implicit aggregation on multi-core architectures

  2025influential citation
• Data-aware Dynamic Execution of Irregular Workloads on Heterogeneous Systems

  2025cites this paper
• Decentralized Stochastic Optimization Over Unreliable Networks via Two-Timescales Updates

  2025cites this paper
• E-3SFC: Communication-Efficient Federated Learning With Double-Way Features Synthesizing

  2025cites this paper
• Scalable AI: Leveraging Cloud Infrastructure for Large-Scale Machine Learning

  2025cites this paper
• Unbiased and Sign Compression in Distributed Learning: Comparing Noise Resilience via SDEs

  2025cites this paper
• Provable Model-Parallel Distributed Principal Component Analysis with Parallel Deflation

  2025cites this paper
• DINA: Toward Determined In-Network Aggregation for Distributed Machine Learning

  2025cites this paper
• Revisiting the In-Network Aggregation in Distributed Machine Learning

  2025cites this paper
• Highly Parallelized Reinforcement Learning Training with Relaxed Assignment Dependencies

  2025cites this paper
• LV-XAttn: Distributed Cross-Attention for Long Visual Inputs in Multimodal Large Language Models

  2025cites this paper
• ByteScale: Communication-Efficient Scaling of LLM Training with a 2048K Context Length on 16384 GPUs

  2025cites this paper
• Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Training

  2025cites this paper
• Training large-scale language models with limited GPU memory: a survey

  2025cites this paper
• AI Work Quantization Model: Closed-System AI Computational Effort Metric

  2025cites this paper
• Distributed Learning over Arbitrary Topology: Linear Speed-Up with Polynomial Transient Time

  2025cites this paper
• DeFT: Mitigating Data Dependencies for Flexible Communication Scheduling in Distributed Training

  2025cites this paper
• Optimizing Distributed Training Approaches for Scaling Neural Networks

  2025cites this paper
• TubiLearn: Predictive Analysis of Tuberculosis Using Machine Learning Algorithms

  2025cites this paper
• Global-QSGD: Allreduce-Compatible Quantization for Distributed Learning with Theoretical Guarantees

  2025cites this paper
• OVERLORD: Ultimate Scaling of DataLoader for Multi-Source Large Foundation Model Training

  2025cites this paper
• eDKM: An Efficient and Accurate Train-Time Weight Clustering for Large Language Models

  2025cites this paper
• bs-scheduler: A Batch Size Scheduler library compatible with PyTorch DataLoaders

  2025cites this paper
• GuardGrid: a high-availability cloud platform for deep learning applications

  2025cites this paper
• Galvatron: An Automatic Distributed System for Efficient Foundation Model Training

  2025cites this paper
• It's just distributed computing: Rethinking AI governance

  2025cites this paper