Stochastic Approximations and Perturbations in Forward-Backward Splitting for Monotone Operators

We investigate the asymptotic behavior of a stochastic version of the forward-backward splitting algorithm for finding a zero of the sum of a maximally monotone set-valued operator and a cocoercive operator in Hilbert spaces. Our general setting features stochastic approximations of the cocoercive operator and stochastic perturbations in the evaluation of the resolvents of the set-valued operator. In addition, relaxations and not necessarily vanishing proximal parameters are allowed. Weak and strong almost sure convergence properties of the iterates is established under mild conditions on the underlying stochastic processes. Leveraging these results, we also establish the almost sure convergence of the iterates of a stochastic variant of a primal-dual proximal splitting method for composite minimization problems.

• Publication year

  2015

• Venue

  arXiv: Optimization and Control

• Publication date

  2015-07-25

• Fields of study

  Mathematics

• Identifiers
  arXiv 1507.07095
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Minimization Methods For Non Differentiable Functions

  2016cited by this paper
• A closer look at consistent operator splitting and its extensions for topology optimization

  2015influential reference
• A stochastic inertial forward–backward splitting algorithm for multivariate monotone inclusions

  2015influential reference
• On the Convergence of the Iterates of the “Fast Iterative Shrinkage/Thresholding Algorithm”

  2015cited by this paper
• On the convergence rate of a forward-backward type primal-dual splitting algorithm for convex optimization problems

  2015influential reference
• A stochastic coordinate descent primal-dual algorithm and applications

  2014cited by this paper
• A forward-backward view of some primal-dual optimization methods in image recovery

  2014cited by this paper
• A Stochastic forward-backward splitting method for solving monotone inclusions in Hilbert spaces

  2014cited by this paper
• Playing with Duality: An overview of recent primal?dual approaches for solving large-scale optimization problems

  2014cited by this paper
• A Proximal Stochastic Gradient Method with Progressive Variance Reduction

  2014cited by this paper
• Iterative Optimization in Inverse Problems

  2014cited by this paper
• A Class of Randomized Primal-Dual Algorithms for Distributed Optimization

  2014cited by this paper
• Compositions and convex combinations of averaged nonexpansive operators

  2014cited by this paper
• On stochastic proximal gradient algorithms

  2014influential reference
• Convergence Analysis of Primal-Dual Algorithms for a Saddle-Point Problem: From Contraction Perspective

  2012cited by this paper
• A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms

  2012cited by this paper
• Variable metric forward–backward splitting with applications to monotone inclusions in duality

  2012cited by this paper
• Monotone Operator Methods for Nash Equilibria in Non-potential Games

  2011cited by this paper
• A consistent algorithm to solve Lasso, elastic-net and Tikhonov regularization

  2011cited by this paper
• A wavelet-based regularized reconstruction algorithm for SENSE parallel MRI with applications to neuroimaging

  2011cited by this paper
• Proximal Algorithms for Multicomponent Image Recovery Problems

  2011influential reference
• Convex Analysis and Monotone Operator Theory in Hilbert Spaces

  2011cited by this paper
• Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators

  2011cited by this paper
• Non-Asymptotic Analysis of Stochastic Approximation Algorithms for Machine Learning

  2011cited by this paper
• A splitting algorithm for dual monotone inclusions involving cocoercive operators

  2011influential reference
• A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science

  2010cited by this paper
• Dualization of Signal Recovery Problems

  2009influential reference
• Efficient Online and Batch Learning Using Forward Backward Splitting

  2009cited by this paper
• A Parallel Splitting Method for Coupled Monotone Inclusions

  2009influential reference
• A variational formulation for frame-based inverse problems

  2007cited by this paper
• Signal Recovery by Proximal Forward-Backward Splitting

  2005cited by this paper
• Solving monotone inclusions via compositions of nonexpansive averaged operators

  2004influential reference
• Stochastic Approximation and Recursive Algorithms and Applications

  2003influential reference
• Finite-Dimensional Variational Inequalities and Complementarity Problems

  2003cited by this paper
• Which Fixed Point Does the Iteration Method Select

  1997cited by this paper
• STABILITY OF THE ITERATION METHOD FOR NON EXPANSIVE MAPPINGS

  1996cited by this paper
• Stochastic Limit Theory

  1994cited by this paper
• ECOLE POLYTECHNIQUE

  1994cited by this paper
• Applications of splitting algorithm to decomposition in convex programming and variational inequalities

  1991cited by this paper
• Probability in Banach Spaces: Isoperimetry and Processes

  1991cited by this paper
• Further applications of a splitting algorithm to decomposition in variational inequalities and convex programming

  1990cited by this paper
• Methods for digital restoration of signals degraded by a stochastic impulse response

  1989cited by this paper
• Adaptive Signal Processing

  1985cited by this paper
• Topics in Finite Element Solution of Elliptic Problems

  1979influential reference
• On the method of generalized stochastic gradients and quasi-Féjer sequences

  1969cited by this paper
• Some methods of stochastic optimization

  1966cited by this paper
• A Stochastic Approximation Method

  1951cited by this paper
• Journal of Machine Learning Research () Submitted 9/12; Published Stochastic Dual Coordinate Ascent Methods for Regularized Loss Minimization

  year unknowncited by this paper
• C Xxxx Society for Industrial and Applied Mathematics Stochastic Quasi-fejér Block-coordinate Fixed Point Iterations with Random Sweeping *

  year unknowncited by this paper

• Characterizations of inexact proximal operators

  2026cites this paper
• Inertial Stochastic Reflected Forward Backward Method with Applications to Traffic Network Problems

  2025influential citation
• Stochastic Generalized Nash Equilibrium Seeking: Reflected Gradient Methods

  2025cites this paper
• On the asymptotic behaviour of stochastic processes, with applications to supermartingale convergence, Dvoretzky's approximation theorem, and stochastic quasi-Fejér monotonicity

  2025cites this paper
• A Geometric Framework for Stochastic Iterations

  2025influential citation
• GraphGrad: Efficient Estimation of Sparse Polynomial Representations for General State-Space Models

  2024cites this paper
• Regret-Optimal Federated Transfer Learning for Kernel Regression with Applications in American Option Pricing

  2023cites this paper
• Algorithmes Gradient-Proximaux Stochastiques

  2023cites this paper
• RandProx: Primal-Dual Optimization Algorithms with Randomized Proximal Updates

  2022cites this paper
• Stochastic Fixed-Point Iterations for Nonexpansive Maps: Convergence and Error Bounds

  2022influential citation
• A Tseng type stochastic forward-backward algorithm for monotone inclusions

  2022cites this paper
• SABRINA: A Stochastic Subspace Majorization-Minimization Algorithm

  2022cites this paper
• Variance reduction for root-finding problems

  2022cites this paper
• A stochastic three-block splitting algorithm and its application to quantized deep neural networks

  2022influential citation
• A stochastic variance reduction algorithm with Bregman distances for structured composite problems

  2021influential citation
• Convergence analysis of the stochastic reflected forward–backward splitting algorithm

  2021cites this paper
• On the computation of equilibria in monotone and potential stochastic hierarchical games

  2021cites this paper
• Primal Dual Interpretation of the Proximal Stochastic Gradient Langevin Algorithm

  2020cites this paper
• Fixed Point Strategies in Data Science

  2020influential citation
• Variance-Reduced Proximal and Splitting Schemes for Monotone Stochastic Generalized Equations

  2020cites this paper
• Variance-Reduced Splitting Schemes for Monotone Stochastic Generalized Equations

  2020cites this paper
• Learning Sparse Low-Precision Neural Networks With Learnable Regularization

  2020cites this paper
• Asymptotic behavior of a dynamical system on a metric space

  2019cites this paper
• Online MR image reconstruction for compressed sensing acquisition in T2* imaging

  2019cites this paper
• A fully stochastic primal-dual algorithm

  2019cites this paper
• The Stochastic Fejér-Monotone Hybrid Steepest Descent Method and the Hierarchical RLS

  2018cites this paper
• Blended coarse gradient descent for full quantization of deep neural networks

  2018cites this paper
• A Probabilistic Incremental Proximal Gradient Method

  2018cites this paper
• BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weights

  2018cites this paper
• On the linear convergence of the stochastic gradient method with constant step-size

  2018cites this paper
• String-averaging incremental stochastic subgradient algorithms

  2018cites this paper
• Analysis of nonsmooth stochastic approximation: the differential inclusion approach

  2018cites this paper
• Linear Convergence of Stochastic Block-Coordinate Fixed Point Algorithms

  2018cites this paper
• Approche de Douglas-Rachford aléatoire par blocs appliquée à la régression logistique parcimonieuse

  2017cites this paper
• Stochastic Three-Composite Convex Minimization

  2017cites this paper
• A random block-coordinate Douglas–Rachford splitting method with low computational complexity for binary logistic regression

  2017cites this paper
• Stochastic proximal-gradient algorithms for penalized mixed models

  2017cites this paper
• Nonasymptotic convergence of stochastic proximal point methods for constrained convex optimization

  2017cites this paper
• On the linear convergence of the stochastic gradient method with constant step-size

  2017cites this paper
• String-averaging incremental subgradient methods for constrained convex optimization problems

  2017cites this paper
• A constant step Forward-Backward algorithm involving random maximal monotone operators

  2017cites this paper
• Nonasymptotic convergence of stochastic proximal point algorithms for constrained convex optimization

  2017cites this paper
• On the linear convergence of the projected stochastic gradient method with constant step-size

  2017cites this paper
• Stochastic quasi-Fejér block-coordinate fixed point iterations with random sweeping II: mean-square and linear convergence

  2017cites this paper
• Link dependent origin-destination matrix estimation: nonsmooth convex optimisation with Bluetooth-inferred trajectories

  2016cites this paper
• String-averaging incremental subgradients for constrained convex optimization with applications to reconstruction of tomographic images

  2016cites this paper
• Stochastic forward-backward and primal-dual approximation algorithms with application to online image restoration

  2016influential citation
• A Primal-Dual Algorithm for Link Dependent Origin Destination Matrix Estimation

  2016cites this paper
• Stochastic Forward Douglas-Rachford Splitting for Monotone Inclusions

  2016influential citation
• Random primal-dual proximal iterations for sparse multiclass SVM

  2016cites this paper
• Stochastic Forward–Backward Splitting for Monotone Inclusions

  2016influential citation
• Dynamical Behavior of a Stochastic Forward–Backward Algorithm Using Random Monotone Operators

  2016cites this paper
• Stochastic Forward–Backward Splitting for Monotone Inclusions

  2016cites this paper
• Two stochastic optimization algorithms for convex optimization with fixed point constraints

  2016cites this paper
• A First-Order Stochastic Primal-Dual Algorithm with Correction Step

  2016cites this paper
• Diffusion stochastic optimization with non-smooth regularizers

  2016cites this paper
• A Stochastic Majorize-Minimize Subspace Algorithm for Online Penalized Least Squares Estimation

  2015cites this paper
• Dynamical Behavior of a Stochastic Forward–Backward Algorithm Using Random Monotone Operators

  2015cites this paper
• Convergence of Stochastic Proximal Gradient Algorithm

  2014cites this paper
• On Perturbed Proximal Gradient Algorithms

  2014cites this paper