Optimal Manufacturing Configuration Selection: Sequential Decision Making and Optimization using Reinforcement Learning

Torayev, Agajan; Abadia, Jose Joaquin Peralta; Martínez-Arellano, Giovanna; Cuesta, Mikel; Chaplin, Jack C; Larrinaga, Felix; Sanderson, David; Arrazola, Pedro José; Ratchev, Svetan

Abstract

In manufacturing, different costs must be considered when selecting the optimal manufacturing configuration. Costs include manufacturing costs, material costs, labor costs, and overhead costs. Optimal manufacturing configurations are those that minimize production criteria, such as costs, production speed, and flexibility, while still meeting the required production levels and quality standards. To find the optimal manufacturing configuration, manufacturers often use a combination of traditional techniques, e.g., mathematical modeling, simulation, and optimization, to evaluate the tradeoffs between different cost factors and identify configurations that provide the best balance between cost and performance. However, these techniques may require long development and simulation time, and/or may require expert knowledge. This paper presents a method for selecting the optimal manufacturing configuration, focusing on cost optimization, using a reinforcement learning (RL) approach for sequential decision-making. The proposed method involves developing a RL environment, requiring lower development and simulation times than traditional techniques, that captures the incurred costs, recurring costs, production rates, and setup times of manufacturing configurations. The problem is then solved using the Proximal Policy Optimization algorithm to identify the configuration that minimizes costs while still meeting the required production levels and quality standards. The effectiveness of the proposed method is validated through a machining process planning case study with multiple cost factors and production constraints. In particular, the machining process plan was developed for an industry-relevant product prototype. The results show that the proposed method can find solutions that are robust to stochastic noise, providing valuable insights for manufacturers looking to optimize manufacturing operations.

Citation

Torayev, A., Abadia, J. J. P., Martínez-Arellano, G., Cuesta, M., Chaplin, J. C., Larrinaga, F., Sanderson, D., Arrazola, P. J., & Ratchev, S. (2023, October). Optimal Manufacturing Configuration Selection: Sequential Decision Making and Optimization using Reinforcement Learning. Presented at 56th CIRP Conference onManufacturing Systems, CIRP CMS ‘23, Cape Town

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 56th CIRP Conference onManufacturing Systems, CIRP CMS ‘23
• Start Date: Oct 24, 2023
• End Date: Oct 26, 2023
• Acceptance Date: Jun 12, 2022
• Online Publication Date: Jan 12, 2024
• Publication Date: 2023
• Deposit Date: Feb 27, 2024
• Publicly Available Date: Feb 27, 2024
• Journal: Procedia CIRP
• Electronic ISSN: 2212-8271
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 120
• Pages: 986-991
• DOI: https://doi.org/10.1016/j.procir.2023.09.112
• Keywords: manufacturing; optimization; decision making; artificial intelligence; machining
• Public URL: https://nottingham-repository.worktribe.com/output/31619305
• Publisher URL: https://www.sciencedirect.com/science/article/pii/S2212827123008442?via%3Dihub
• Additional Information: This article is maintained by: Elsevier; Article Title: Optimal Manufacturing Configuration Selection: Sequential Decision Making and Optimization using Reinforcement Learning; Journal Title: Procedia CIRP; CrossRef DOI link to publisher maintained version: https://doi.org/10.1016/j.procir.2023.09.112; Content Type: article; Copyright: © 2023 The Author(s). Published by Elsevier B.V.