Data-driven development of advanced controllers for complex reaction systems with minimal prior information

Abstract

In the realm of complex reactive systems where full knowledge of ongoing reactions is unattainable, the adoption of data-driven inferential models based on mixture spectra has gained significant traction. Spectra-based online monitoring has shown promise due to the rapidity, non-invasiveness, non-destructiveness, and cost-effectiveness in spectral analysis. This study aims to develop advanced controllers for such complex reactive systems in the absence of ground truth information and subsequently compare their performances. To achieve this objective, a comprehensive suite of tools, including spectral deconvolution, Bayesian networks, neural ordinary differential equations (ODEs), long short-term memory (LSTM), model predictive control (MPC), and reinforcement learning are employed to transform spectra into actionable control strategies. The initial phase of the research focuses on establishing a model-based control framework through the utilization of spectral deconvolution and Bayesian networks, particularly in scenarios where ground truth knowledge is limited or the system dynamics are complex. Spectral deconvolution untangles pseudo component spectra and their corresponding concentration profiles from mixture spectra. These deconvoluted spectra serve as the Bayesian network’s inputs, effectively identifying potential reaction networks within the system. Concurrently, neural ODEs leverage the concentration profiles obtained from spectral deconvolution to extract rate law parameters and facilitate step-ahead concentration predictions. This holistic approach results in a comprehensive rate-law-based kinetic model that captures the reaction system’s dynamics. Two modeling approaches are employed and compared: a data-driven LSTM and a physics-driven greybox model utilizing Neural ODE. While the LSTM model operates as a black box, providing step-ahead concentration predictions, the Neural ODE model represents a grey-box approach incorporating first principles, also generating step-ahead predictions. The aim is to evaluate the performance of these approaches, contrasting the efficacy of the data-driven black box model (LSTM) with the physics-driven grey-box model (Neural ODE). In the latter phase of the study, reinforcement learning-based techniques are leveraged to design a model-free controller with a focus on optimizing the selectivity of desired products, like in MPC with neural ODE as model/environment. For future work, the focus will be on leveraging spectra corresponding to specific wavenumber ranges that are indicative of the functional groups associated with target products. This strategy diverges from previous approaches, such as the deconvolution pathway that emphasized modeling the kinetics. Instead, the plan is to adopt a model that utilizes mixture spectra as inputs. This model, in its control segment, will be designed to incentivize the agent or controller to prioritize selectivity towards certain products and/or wavenumber ranges. This methodology enables the system to refine its control strategy by relying solely on spectral data. This is particularly beneficial in situations where a comprehensive understanding of the system’s dynamics is not available, thus circumventing the need to develop detailed kinetic models. In conclusion, this work harnesses a range of advanced modelling and control methodologies to translate spectral data into actionable control strategies for complex reactive systems. The efficacy of the developed controllers is demonstrated through a simulation environment of a CSTR aimed at maximizing the selectivity of a desired species, thereby achieving the desired overall system performance.