Physics-informed neural networks for data-efficient & accurate learning of physical systems

Abstract

Traditionally, Scientific Computing and Computer Aided Engineering software use numerical solvers for simulating physics-based models. Despite their high accuracy, numerical solvers can become very computationally intensive and often are impractical to be applied in real time applications in Hardware-in-the-Loop systems (HiL), Electronic Control Units (ECU) or modern edge devices (Microcontrollers). On the other hand, many machine learning models such as Artificial Neural Networks are universal function approximators with very small inference, time and memory footprint. For the above reasons, modern scientific computing makes extensive use of Machine Learning for speeding up simulations or optimization processes. However, this use is limited by the presence of measurement or simulation data. In many cases, collecting measurements is not an option due to the high experimental costs, while in the case of simulation data, the need to run expensive in time simulations will often arise. Scientific Machine Learning or Physics-Informed Machine Learning tries to tackle the lack of training data by incorporating physics-based laws into the training process of machine learning models. More specifically, Physics Informed Neural Networks (PINNs) are a type of Neural Networks that are trained not only on data, if data are available, as it is usually the case in deep learning, but also on the model of the differential equations describing the underlying laws of physics, which makes them extremely accurate and data efficient. This ability to train a Neural Network in a non-arbitrary unsupervised way is a real breakthrough for the field of Machine Learning in general. The aim of this MSc thesis is to apply the PINNs methodology in solving a variety of benchmark dynamical systems. We experimentally verify the ability of PINNs to solve standard benchmark problems such as Burgers equation and Poisson equation. We explore the borders of this technology by applying PINNs in challenging Fluid Dynamics problems, such as the NavierStokes equations. In Lid-Driven Cavity Flow, our trained PINNs demonstrate competitive performance in terms of accuracy when compared to established numerical solvers. Furthermore, they produce more precise results than those reported in a relevant PINN reference paper [1], all while utilizing significantly smaller neural network architectures. This becomes feasible by proposing and applying alternative optimization schemes. In the case of the Static Piston Flow problem, where PINNs failed to find the solution because of high nonlinearity and increased turbulence, we solve the problem using the classic Supervised Learning (SL) approach, which applies similar in size NN architectures, and we provide comparative results for the various optimizers used. The high accuracy achieved using the SL is a clear indication that the reason of PINNs’ failure in that case was not the learning capability of the Neural Network but the complexity of the optimization problem itself. Finally, after extensive search in the bibliography, several candidate solutions have been gathered and are presented that could help to expand the limits of this technology and make it applicable to real world applications.