In general, hybrid reasoning aims to combine the statistical power of machine learning models and the mechanistic strength of physics-based models. At the implementation level, hybridization is achieved by incorporating physics-based bias into the learning system at different levels. Accordingly, hybrid approaches can be categorized into three main types: observational bias, learning bias, and inductive bias

Inductive bias focuses on utilizing prior knowledge about a dynamic system, supported by physics, to modify the computational graph of a machine learning model and enforce consistency with that prior knowledge. Artificial neural networks are commonly used within a customized computational graph for this purpose. The modification entails incorporating additional computational steps, including mathematical operations, to apply the physics-derived prior knowledge. By integrating physics-based constraints into the computational graph, the machine learning model can capture the underlying principles and relationships of the dynamic system more accurately. Mathematical operations, such as differentiation, can enforce the model to approximate differential equations that represent many physics-based phenomena. The reduction of the hypothesis space and constraint enforcement ensure that the learned model not only fits the available data but also adheres to the fundamental physics of the domain.

Our team developed a hybrid model based on the inductive bias for modeling multi-dimensional dynamics of a coupled nonlinear dynamical system. This approach leverages principles from classical mechanics, such as the Euler-Lagrange and Hamiltonian formalisms, to facilitate the process of learning from data. The hybrid model incorporates single or multiple artificial neural networks within a customized computational graph designed based on the physics of the problem. The customization minimizes the potential of violating the underlying physics and maximizes the efficiency of information flow within the model. The capabilities of this approach have been investigated for various multidimensional modeling scenarios using different configurations of a coupled nonlinear dynamical system. It has been demonstrated that, in addition to improving modeling criteria such as accuracy and consistency with physics, this approach provides additional modeling benefits. The hybrid model implements a physics-based architecture, enabling the direct alteration of both conservative and non-conservative components of the dynamics. This allows for an expansion in the model's input dimensionality and optimal allocation of input variable effects on conservative or non-conservative components of dynamics.

The below figure illustrates the computational graphs of two different neural network models: a multidimensional standard neural network and a multidimensional Hamiltonian neural network. The second network is a multidimensional dissipative Hamiltonian neural network. In this model, the mapping from the position and momentum to the associated time derivatives is indirectly developed in the learning process by two sub-networks. The intermediate variables parametrize the functions (𝐻 and 𝐷 ) which represent the conservative and dissipative energy. These variables go through in-graph derivatives, which return the associated forces. Finally, the contributions of the predictions of the conservative component (noted by the subscript ℎ in the figure) and those of the dissipative component (noted by the subscript 𝑑 in the figure) are added to fully reconstruct the dynamics of the system.

Computational graph of (a) a standard neural network, (b) a Dissipative Hamiltonian neural network

The customized computational graph of the multidimensional dissipative Hamiltonian neural network can be further modified for dynamical system modeling purposes. As a case in point, deconstructing dynamics into conservative and neoconservative components allows for allocating the effects of an input variable solely on conservative or non-conservative components. This segmentation provides for optimal flow of information in the network guided by physics of the dynamical system. For example, in a multi-dimensional coupled system in which the coupling is through stiffness, the physics of the problems implies a direct effect of the coupling coefficient on the conservative component of dynamics. The below figure illustrates the model designed based on this concept, with the additional input noted by 𝑐 in the figure.

The following figure represents multidimensional dissipative Hamiltonian neural network with an additional input variable to customize the conservative component.The below figure shows a graphical summary of this data collection scenario for a value of alpha (α) applied to trajectories associated with x and y degrees of freedom of a coupled dynamic system. The parameter alpha (α) represents the additional input. The vector field of each oscillator is partitioned into n two-dimensional grid of points within the range of [qx-min, qx-max] and [px-min, px-max] for the x degree of freedom and [qy-min, qy-max] and [py-min, py-max] for the y degree of freedom for each value of alpha (α).

Accordingly, each point on the grid is represented by a pair of coordinates such as [qxij, pxij] for the x degree of freedom and [qyij, pyij] for the y degree of freedom, along with the variable αi. These points, along with this variable, serve as the inputs of the model, while the associated time derivatives, which are [q̇xij, ṗxij] and [q̇yij, ṗyij], are considered as the outputs. Hence, each sample includes [αi, qxij, pxij, qyij, pyij] as the input and [q̇xij, ṗxij, q̇yij, ṗyij] as the output. This figure depicts two arbitrary trajectories in the vector field of the dynamical system with x and y degrees of freedom, represented as trx and try, at a specific value of the variable α denoted as αi. Parts (a) and (c) of this figure display the collection of input and output samples from these trajectories, while Part (b) illustrates the concatenation of dynamic variables [qxij, pxij, qyij, pyij] and αi for training the model.

The results indicate that the hybrid model reduces the likelihood of violating the underlying principles of physics and enhances the efficiency of information flow within the model. In addition, it enables the adaptation of the model to various conditions such as conservative and non-conservative. Additionally, the customization allows for the increase in the number of inputs in the hybrid model and the assignment of effects of the input variables to either the conservative or non-conservative components of the dynamics.

The following figure represents graphical summary of the data collection.The performance of this approach is evaluated in various modeling scenarios using different configurations of a coupled nonlinear dynamical system, based on statistical and physics-based measures. The results indicate that the hybrid model can outperform the pure machine learning model, depending on the modeling scenario up to 78% and 79% in statistical and physics-based measures, respectively. The implicit decomposition of the dynamical system into conservative and non-conservative components opens up additional modeling possibilities, such as developing an adaptive hybrid model that can compensate for the parametric drift of the dynamical system. The control over information flow within the model enables direct tuning of either the conservative or non-conservative components, depending on the source of the parametric drift. This capability allows the model to adapt and preserve its performance. This is a subject of our ongoing research.