Abstract
Manufacturing system is more interconnected and transparent with the deployment of distributed sensors and automatic machinery, as well as data storage and processing capabilities due to increasing availability of computing resources. Machine learning techniques are very promising in gaining useful insights from the huge volume of real-time data to facilitate system performance analysis and control decision makings. Despite of exciting advances in machine learning research and application in the past decade, it remains a challenging task to apply those techniques in the context of manufacturing industry. The expected improvements in productivity, quality and efficiency are still hampered by the salient gaps in real-time system modeling, system performance evaluation and prediction, and theory and algorithms for integrated decision making and optimization in the manufacturing domain.
In particular, reinforcement learning (RL) and multi-agent reinforcement learning (MARL), which aim at understanding the dynamics of the process/system and finding the optimization strategy through interactions with the environment, have opened up a new research avenue of the intended system performance enhancement without a rigid rulebook. However, manufacturing system is a complex engineering system with very high stochasticity and nonlinearity as well as great varieties in processes/products and scales. The system dynamics is deeply coupled with individual machines and processes, and constantly evolving due to not only internal factors, e.g., machine and process constraints, but also external circumstances like customer demands. This dissertation demonstrates a systematic way to use domain knowledge and systematic understanding of the manufacturing system to formulate typical control problems in RL/MARL framework in the manufacturing domain. In this dissertation, we start from analytical system modeling based on basic physics and then derive system properties, which are further used to guide the problem formulation and algorithm implementation in a variety of significant prediction and control problems. The dissertation contributes to the body of research in manufacturing systems regarding the following aspects:
(1) A data-enabled system model for multi-product manufacturing system is established based on basic physic law, i.e., the conservation of the flow. The product-dependent cycle time and tool setup time are considered in the model. It closely connects the data collected from distributed sensors to the system states. The model shed lights on knowledge-guided machine learning problems formulation and solution.
(2) A hybrid framework combining deep learning and system modeling is developed to predict product completion time, which is critical to downstream tasks including production scheduling. Guided by system properties, a recurrent sequence in the prediction problem is discovered, and hence Long Short-Term Memory, a variant of Recurrent Neural Network, is applied.
(3) The preventive maintenance control problem is tackled using deep RL techniques. It demonstrates the formulation of manufacturing system control problems in the RL/MARL frameworks. By implementing both deep RL and deep MARL algorithms, it covers the preventive maintenance decision making in all spectra of manufacturing systems regarding the sizes and maintenance options.
(4) An innovative multi-agent control framework that integrates multiple levels of a manufacturing industry, including system level, process level and machine level, are proposed with the aim to optimize system performance considering both productivity and quality. The graph model and graph neural networks are applied to encode and integrate information across multiple levels and machines. Recursive Bayesian Estimation is applied to graph node feature engineering.
