Multi-agent reinforcement learning for flexible shop scheduling problem: a survey

This paper presents a systematic and comprehensive review of multi-agent reinforcement learning (MARL) methodologies and their applications in addressing the flexible shop scheduling problem (FSSP), a fundamental yet challenging optimization paradigm in contemporary manufacturing systems. While conventional optimization approaches exhibit limitations in handling the inherent multi-resource constraints, dynamics and stochastic characteristics of real-world FSSP scenarios, MARL has emerged as a promising alternative framework, particularly due to its capability to effectively manage complex, decentralized decision-making processes in dynamic environments. Through a rigorous analytical framework, this study synthesizes and evaluates the current state-of-the-art MARL implementations in FSSP contexts, encompassing critical aspects such as problem formulation paradigms, agent architectural designs, learning algorithm frameworks, and inter-agent coordination mechanisms. We conduct an in-depth examination of the fundamental challenges inherent in MARL applications to FSSP, including the optimization of state-action space representations, the design of effective reward mechanisms, and the resolution of scalability constraints. Furthermore, this review provides a comparative analysis of diverse MARL paradigms, including centralized training with decentralized execution, fully decentralized approaches, and hierarchical methodologies, critically evaluating their respective advantages and limitations within the FSSP domain. The study culminates in the identification of significant research gaps and promising future research directions, with particular emphasis on theoretical foundations and practical implementations. This comprehensive review serves as an authoritative reference for researchers and practitioners in the field, providing a robust theoretical foundation and practical insights for advancing the application of MARL in flexible shop scheduling and related manufacturing optimization domains. The findings presented herein contribute to the broader understanding of intelligent manufacturing systems and computational optimization in Industry 4.0 contexts.

• Publication year

  2025

• Venue

  Frontiers in Industrial Engineering

• Publication date

  2025-07-31

• Fields of study

  Not labeled

• Identifiers
  DOI 10.3389/fieng.2025.1611512
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Dynamic flexible flow shop scheduling via cross-attention networks and multi-agent reinforcement learning

  2025cited by this paper
• Structural entropy-based scheduler for job planning problems using multi-agent reinforcement learning

  2025cited by this paper
• Real-Time Scheduling for Flexible Job Shop With AGVs Using Multiagent Reinforcement Learning and Efficient Action Decoding

  2025cited by this paper
• A multi-agent reinforcement learning based scheduling strategy for flexible job shops under machine breakdowns

  2025cited by this paper
• End-to-End Multitarget Flexible Job Shop Scheduling With Deep Reinforcement Learning

  2025cited by this paper
• Multi-Objective Scheduling for Green Flexible Assembly Job-Shop System via Multi-Agent Deep Reinforcement Learning With Game Theory

  2025cited by this paper
• Hierarchical multi-agent deep reinforcement learning for dynamic flexible job-shop scheduling with transportation

  2025cited by this paper
• Multi-Agent Reinforcement Learning for Distributed Flexible Job Shop Scheduling With Random Job Arrival

  2025cited by this paper
• Dynamic flexible scheduling with transportation constraints by multi-agent reinforcement learning

  2024cited by this paper
• Dynamic scheduling mechanism for intelligent workshop with deep reinforcement learning method based on multi-agent system architecture

  2024cited by this paper
• Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

  2024cited by this paper
• Reentrant hybrid flow shop scheduling with stockers in automated material handling systems using deep reinforcement learning

  2024cited by this paper
• From Nash Q-learning to nash-MADDPG: Advancements in multiagent control for multiproduct flexible manufacturing systems

  2024cited by this paper
• Multi-Agent Scheduler for the Continuous Dynamic Flexible Job Shop Scheduling Problem

  2024cited by this paper
• Study on the application of single-agent and multi-agent reinforcement learning to dynamic scheduling in manufacturing environments with growing complexity: Case study on the synthesis of an industrial IoT Test Bed

  2024cited by this paper
• Dynamic flexible job-shop scheduling by multi-agent reinforcement learning with reward-shaping

  2024cited by this paper
• Knowledge graph-enhanced multi-agent reinforcement learning for adaptive scheduling in smart manufacturing

  2024cited by this paper
• Dynamic Hybrid Flow Shop Scheduling in Multi-Agent Manufacturing Systems via Federated Transfer Learning

  2024cited by this paper
• Learning-Based Genetic Algorithm to Schedule an Extended Flexible Job Shop

  2024cited by this paper
• Collaborative dynamic scheduling in a self-organizing manufacturing system using multi-agent reinforcement learning

  2024cited by this paper
• Hierarchical Reinforcement Learning in Multi-Domain Elastic Optical Networks to Realize Joint RMSA

  2023cited by this paper
• Multi-Agent Reinforcement Learning for Extended Flexible Job Shop Scheduling

  2023cited by this paper
• A Reinforcement-Learning-Based 3-D Estimation of Distribution Algorithm for Fuzzy Distributed Hybrid Flow-Shop Scheduling Considering On-Time-Delivery

  2023cited by this paper
• A multi-agent double Deep-Q-network based on state machine and event stream for flexible job shop scheduling problem

  2023cited by this paper
• Multi-objective FJSP Based on Multi-agent Reinforcement Learning Algorithm

  2023cited by this paper
• Counterfactual-attention multi-agent reinforcement learning for joint condition-based maintenance and production scheduling

  2023cited by this paper
• Applying Multi-agent Reinforcement Learning and Graph Neural Networks to Flexible Job Shop Scheduling Problem

  2023cited by this paper
• Integration of deep reinforcement learning and multi-agent system for dynamic scheduling of re-entrant hybrid flow shop considering worker fatigue and skill levels

  2023cited by this paper
• Dynamic production scheduling towards self-organizing mass personalization: A multi-agent dueling deep reinforcement learning approach

  2023cited by this paper
• Multi-Task Multi-Agent Reinforcement Learning for Real-Time Scheduling of a Dual-Resource Flexible Job Shop with Robots

  2023cited by this paper
• Agent-based simulation and optimization of hybrid flow shop considering multi-skilled workers and fatigue factors

  2023cited by this paper
• Traffic Signal Control System Using Deep Reinforcement Learning With Emphasis on Reinforcing Successful Experiences

  2022cited by this paper
• Real-Time Scheduling for Dynamic Partial-No-Wait Multiobjective Flexible Job Shop by Deep Reinforcement Learning

  2022cited by this paper
• Deep reinforcement learning for dynamic scheduling of a flexible job shop

  2022cited by this paper
• End-to-End Autonomous Driving With Semantic Depth Cloud Mapping and Multi-Agent

  2022cited by this paper
• Oracle-Guided Deep Reinforcement Learning for Large-Scale Multi-UAVs Flocking and Navigation

  2022cited by this paper
• A multi-action deep reinforcement learning framework for flexible Job-shop scheduling problem

  2022cited by this paper
• A Cooperation Graph Approach for Multiagent Sparse Reward Reinforcement Learning

  2022cited by this paper
• Using Multi-Agent Deep Reinforcement Learning For Flexible Job Shop Scheduling Problems

  2022cited by this paper
• Improving scalability of multi-agent reinforcement learning with parameters sharing

  2022cited by this paper
• Hierarchical Architecture for Multi-Agent Reinforcement Learning in Intelligent Game

  2022cited by this paper
• A Reinforcement Learning Approach for Flexible Job Shop Scheduling Problem With Crane Transportation and Setup Times

  2022cited by this paper
• Multi-agent reinforcement learning based on graph convolutional network for flexible job shop scheduling

  2022cited by this paper
• DeepMAG: Deep reinforcement learning with multi-agent graphs for flexible job shop scheduling

  2022cited by this paper
• Independent double DQN-based multi-agent reinforcement learning approach for online two-stage hybrid flow shop scheduling with batch machines

  2022cited by this paper
• A review of the applications of multi-agent reinforcement learning in smart factories

  2022cited by this paper
• Global Reward Design for Cooperative Agents to Achieve Flexible Production Control under Real-time Constraints

  2021cited by this paper
• Integration of the A2C Algorithm for Production Scheduling in a Two-Stage Hybrid Flow Shop Environment

  2021cited by this paper
• Utilizing Multi-Agent Deep Reinforcement Learning For Flexible Job Shop Scheduling Under Sustainable Viewpoints

  2021cited by this paper
• Multi-Agent Reinforcement Learning for Energy Harvesting Two-Hop Communications With a Partially Observable System State

  2021cited by this paper
• Reinforcement learning applications to machine scheduling problems: a comprehensive literature review

  2021cited by this paper
• Dynamic multi-objective scheduling for flexible job shop by deep reinforcement learning

  2021cited by this paper
• Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning

  2020cited by this paper
• Multi-agent system and reinforcement learning approach for distributed intelligence in a flexible smart manufacturing system

  2020cited by this paper
• Weighted QMIX: Expanding Monotonic Value Function Factorisation

  2020cited by this paper
• Evolving Scheduling Heuristics via Genetic Programming With Feature Selection in Dynamic Flexible Job-Shop Scheduling

  2020cited by this paper
• The Application of Deep Reinforcement Learning to Distributed Spectrum Access in Dynamic Heterogeneous Environments With Partial Observations

  2020cited by this paper
• Reinforcement learning versus evolutionary computation: A survey on hybrid algorithms

  2019cited by this paper
• Multi-Agent Reinforcement Learning for Job Shop Scheduling in Flexible Manufacturing Systems

  2019cited by this paper
• Rewards Prediction-Based Credit Assignment for Reinforcement Learning With Sparse Binary Rewards

  2019cited by this paper
• A review of cooperative multi-agent deep reinforcement learning

  2019cited by this paper
• Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms

  2019cited by this paper
• QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning

  2019cited by this paper
• Solving the flexible job shop scheduling problem with sequence-dependent setup times

  2018cited by this paper
• Zero Shot Transfer Learning for Robot Soccer

  2018cited by this paper
• Review and classification of hybrid shop scheduling

  2018cited by this paper
• Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications

  2018cited by this paper
• Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning

  2017cited by this paper
• A distributed approach solving partially flexible job-shop scheduling problem with a Q-learning effect

  2017cited by this paper
• Cooperative Multi-agent Control Using Deep Reinforcement Learning

  2017cited by this paper
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning

  2016cited by this paper
• Learning Multiagent Communication with Backpropagation

  2016cited by this paper
• Human-level control through deep reinforcement learning

  2015cited by this paper
• Game Theory and Multi-agent Reinforcement Learning

  2012cited by this paper
• The hybrid flow shop scheduling problem

  2010cited by this paper
• Dynamic Decision Model in Evolutionary Games Based on Reinforcement Learning

  2009cited by this paper
• A hybrid genetic and variable neighborhood descent algorithm for flexible job shop scheduling problems

  2008cited by this paper
• A Comprehensive Survey of Multiagent Reinforcement Learning

  2008cited by this paper
• Flexible job-shop scheduling with routing flexibility and separable setup times using ant colony optimisation method

  2007cited by this paper
• Total completion time minimization in a computer system with a server and two parallel processors

  2005cited by this paper
• REINFORCEMENT LEARNING: AN INTRODUCTION by Richard S. Sutton and Andrew G. Barto, Adaptive Computation and Machine Learning series, MIT Press (Bradford Book), Cambridge, Mass., 1998, xviii + 322 pp, ISBN 0-262-19398-1, (hardback, £31.95).

  1999cited by this paper
• Reinforcement Learning: A Survey

  1996cited by this paper
• Knowledge-Based Systems

  year unknowncited by this paper

• Arc-based formulation and GRASP-enhanced iterated greedy algorithm for identical parallel machine scheduling with a common server

  2026cites this paper
• Safety-Aware Multi-Agent Deep Reinforcement Learning for Adaptive Fault-Tolerant Control in Sensor-Lean Industrial Systems: Validation in Beverage CIP

  2026cites this paper
• Industrial Scheduling in the Digital Era: Challenges, State-of-the-Art Methods, and Deep Learning Perspectives

  2025cites this paper