Multi-Agent Reinforcement Learning for Real-Time Dynamic Production Scheduling in a Robot Assembly Cell

As industry rapidly shifts towards mass personalisation, the need for a decentralised multi-agent system capable of dynamic flexible job shop scheduling (FJSP) is evident. Traditional heuristic and meta-heuristic scheduling methods cannot achieve satisfactory results and have limited application to static environments. Recent Reinforcement Learning (RL) approaches that consider dynamic FJSP, lack flexibility and autonomy as they use a single-agent centralised model, assuming global observability. As such, we propose a Multi-Agent Reinforcement Learning (MARL) system for scheduling dynamically arriving assembly jobs in a robot assembly cell. We applied a Double DQN-based algorithm and proposed a generalised observation, action and reward design for the dynamic FJSP setting. Using a centralised training phase, each agent (i.e., robot) in the assembly cell executes decentralised scheduling decisions based on local observations. Our solution demonstrated improved performance against rule-based heuristic methods, for optimising makespan. We also reported the impact of different observation sizes of each agent on optimisation performance.

• Publication year

  2022

• Venue

  IEEE Robotics and Automation Letters

• Publication date

  2022-07-01

• Fields of study

  Computer Science, Engineering

• Identifiers
  DOI 10.1109/lra.2022.3184795
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• No references are available for this paper.

• Efficient scheduling for fixed-type multi-robot collaborative problem in flexible job shop

  2026cites this paper
• Recent progress and challenges of key technologies in robotic assembly

  2026cites this paper
• Development of Inventory Policy Using Demand Driven Material Requirements Planning (Ddmrp) and Probabilistic Inventory Model: A Case Study on the Water Tank Industry in Indonesia

  2026cites this paper
• From insight to action: Embodied multi-agent system integrating vision language model for digital twin-assisted human-robot collaborative assembly

  2026cites this paper
• A Digital-Twin-Enabled AI-Driven Adaptive Planning Platform for Sustainable and Reliable Manufacturing

  2026cites this paper
• Decentralised task planning and motion coordination for scalable multi-robot collaborative manufacturing

  2026cites this paper
• Task Allocation with Communication Coordination in UAV Swarms via Asynchronous Multi-Objective Policy Optimization

  2026cites this paper
• A comprehensive review on multi robot task assignment and path planning: Operational constraints, challenges, and potentials of deep reinforcement learning

  2026cites this paper
• Towards greater resilience: A systematic review of dynamic shop floor scheduling in industry 5.0

  2026cites this paper
• Service Placement in Dynamic Multi-AGV Environments for Minimized Energy Consumption

  2026cites this paper
• Large language model driven adaptive deep reinforcement learning with dynamic definition refinement for flexible job shop scheduling problem

  2026cites this paper
• Machine learning-driven solutions for sustainable and dynamic flexible job shop scheduling under worker absences and renewable energy variability

  2026cites this paper
• Resilient Non-Fragile Consensus Control of MASs under Stochastic Communication Link Faults

  2025cites this paper
• Evolutionary Trainer-Based Deep Q-Network for Dynamic Flexible Job-Shop Scheduling

  2025cites this paper
• Learning Individual Potential-Based Rewards in Multiagent Reinforcement Learning

  2025cites this paper
• A multi-agent reinforcement learning based scheduling strategy for flexible job shops under machine breakdowns

  2025cites this paper
• An intelligent manufacturing system based on a recursive control structure

  2025cites this paper
• A Hierarchical Multi-Action Deep Reinforcement Learning Method for Dynamic Distributed Job-Shop Scheduling Problem With Job Arrivals

  2025cites this paper
• Static scheduling method for aircraft flat-tail assembly production based on improved bi-level genetic algorithm

  2025cites this paper
• Solving quay wall allocation problems based on deep reinforcement learning

  2025cites this paper
• Real-time response to machine failures in self-organizing production execution using multi-agent reinforcement learning with effective samples

  2025cites this paper
• Meta-learning for dynamic multi-robot task scheduling

  2025cites this paper
• Deep reinforcement learning for job shop scheduling problems: A comprehensive literature review

  2025cites this paper
• Flexible Job Shop Scheduling with Job Precedence Constraints: A Deep Reinforcement Learning Approach

  2025cites this paper
• Multi-Task Multi-Agent Reinforcement Learning via Skill Graphs

  2025cites this paper
• MTEAL: Network routing optimization of SD-WAN traffic engineering integrating multi-dimensional QoS metrics

  2025cites this paper
• Graph reinforcement learning based dynamic scheduling for human-machine symbiosis manufacturing considering multi-type disturbance events

  2025cites this paper
• Impacts of Heterogeneous Autonomous Mobile Robots (AMRs) on Smart Manufacturing

  2025cites this paper
• Human-robot collaborative disassembly task planning for retired power battery based on Stackelberg game and multi-agent deep reinforcement learning

  2025cites this paper
• An Improved NSGA-II for Three-Stage Distributed Heterogeneous Hybrid Flowshop Scheduling with Flexible Assembly and Discrete Transportation

  2025cites this paper
• A surrogate model-driven assembly coordination framework for aircraft components based on cooperative multi-agent deep reinforcement learning

  2025cites this paper
• Dynamic green multi-objective scheduling for flexible job shop by deep reinforcement learning

  2025cites this paper
• Reinforcement Learning for Flexibility and Efficient Production

  2025cites this paper
• Communication-Aware Consensus-Based Distributed Task Allocation Based on Asynchronous Reinforcement Learning

  2025cites this paper
• Deep Reinforcement Learning in a Search-Matching Model of Labor Market Fluctuations

  2025cites this paper
• Large Model Scheduling Method in Collaborative Decision Making of Intelligent Agents

  2025cites this paper
• Bridging the Reality Gap: Communication-Aware Task Allocation with Multi-Objective Asynchronous Policy Learning

  2025cites this paper
• Design of an Improved Hybrid Controller for Reachable Set Estimation of Multi-Agent Systems

  2025cites this paper
• A Review of Production Scheduling with Artificial Intelligence and Digital Twins

  2025cites this paper
• Dynamic Human-Robot Collaboration via Hierarchical Reinforcement Learning with Multimodal Intent Prediction and Safety Guarantees

  2025cites this paper
• Energy-Saving Control Strategy for Multisleep States of CNC Machine Tool Considering Components Priority

  2024cites this paper
• Learning to schedule dynamic distributed reconfigurable workshops using expected deep Q-network

  2024cites this paper
• Multi agent reinforcement learning for online layout planning and scheduling in flexible assembly systems

  2024cites this paper
• Stackelberg Game-Theoretic Learning for Collaborative Assembly Task Planning

  2024cites this paper
• A Double Deep Q-Network framework for a flexible job shop scheduling problem with dynamic job arrivals and urgent job insertions

  2024cites this paper
• Towards practicality: Navigating challenges in designing predictive-reactive scheduling

  2024cites this paper
• Edge AI: A Taxonomy, Systematic Review and Future Directions

  2024cites this paper
• Deep Reinforcement Learning-Based Dynamic Reconfiguration Planning for Digital Twin-Driven Smart Manufacturing Systems With Reconfigurable Machine Tools

  2024cites this paper
• Smart scheduling for next generation manufacturing systems: a systematic literature review

  2024cites this paper
• Knowledge graph-enhanced multi-agent reinforcement learning for adaptive scheduling in smart manufacturing

  2024influential citation
• iFANnpp: Nuclear Power Plant Digital Twin for Robots and Autonomous Intelligence

  2024cites this paper
• Dynamic flexible job-shop scheduling by multi-agent reinforcement learning with reward-shaping

  2024cites this paper
• Review on ensemble meta-heuristics and reinforcement learning for manufacturing scheduling problems

  2024cites this paper
• Multi-Agent Scheduler for the Continuous Dynamic Flexible Job Shop Scheduling Problem

  2024influential citation
• Deep reinforcement learning-based dynamic scheduling for resilient and sustainable manufacturing: A systematic review

  2024cites this paper
• A literature review of reinforcement learning methods applied to job-shop scheduling problems

  2024cites this paper
• Application of Inhomogeneous QMIX in Various Architectures to Solve Dynamic Scheduling in Manufacturing Environments

  2024cites this paper
• Multi-Agent Deep Reinforcement Learning for Multi-Robot Applications: A Survey

  2023cites this paper
• Exploring the Impact of Machine Learning on Real-Time Data Analysis

  2023cites this paper
• TPMARL-MSS Thought-Perception Multi-Agent Reinforcement Learning Meeting Scheduling System

  2023cites this paper
• Intelligent scheduling in MES systems for Industry 4.0 - a systematic review of the scientific literature

  2023cites this paper
• Hybrid genetic algorithm with variable neighborhood search for flexible job shop scheduling problem in a machining system

  2023cites this paper
• Digital twin-based multi-level task rescheduling for robotic assembly line

  2023cites this paper
• More Like Real World Game Challenge for Partially Observable Multi-Agent Cooperation

  2023cites this paper
• Dynamic production scheduling towards self-organizing mass personalization: A multi-agent dueling deep reinforcement learning approach

  2023cites this paper
• Multi-Task Multi-Agent Reinforcement Learning for Real-Time Scheduling of a Dual-Resource Flexible Job Shop with Robots

  2023cites this paper
• Counterfactual-attention multi-agent reinforcement learning for joint condition-based maintenance and production scheduling

  2023cites this paper
• Improved Chimp Optimization Algorithm for Matching Combinations of Machine Tool Supply and Demand in Cloud Manufacturing

  2023cites this paper
• A Survey of AI-enabled Dynamic Manufacturing Scheduling: From Directed Heuristics to Autonomous Learning

  2023influential citation
• Deep reinforcement learning in smart manufacturing: A review and prospects

  2023cites this paper
• Research on an Adaptive Real-Time Scheduling Method of Dynamic Job-Shop Based on Reinforcement Learning

  2022cites this paper
• A review of the applications of multi-agent reinforcement learning in smart factories

  2022cites this paper
• Hierarchical Reinforcement Learning for Multi-Objective Real-Time Flexible Scheduling in a Smart Shop Floor

  2022cites this paper
• Reinforcement Learning for Control and Decision-making

  year unknowncites this paper
• on Multi-agent Reinforcement Learning for Control and Decision-making

  year unknowncites this paper