Hybridising standard reduced-order modelling methods with interpretable sparse neural networks for real-time patient-specific lung simulations
VPH 2024 - Stuttgart
September 4, 2024
Lung poro-mechanics
\[ \begin{cases} \boldsymbol{\nabla}\cdot \mathcal{\boldsymbol{\sigma}}+ \boldsymbol{f} = 0 \qquad & \mathrm{in} \ \Omega \\ \mathcal{\boldsymbol{\sigma}}\cdot \boldsymbol{n} = \boldsymbol{F} & \text{on } \partial \Omega_{N} \\ \boldsymbol{u} = \boldsymbol{u}_d \qquad \qquad & \text{on } \partial \Omega_{d} \\ % \sigm = \ftensor{C}\left(\para\right):\eps(\vect{u}) & \mathrm{in} \ \Omega \end{cases} \label{eq:MechPb} \]
Behaviour
- \(\boldsymbol{\Sigma} = \frac{\partial \Psi}{\partial \boldsymbol{E}}\)
- Non-linear finite strain poro-mechanics (Patte et al., 2022)
- \(\boldsymbol{\Sigma}\) Second Piola-Kirchhoff stress tensor
- \(\boldsymbol{E}\) is the Green-Lagrange strain tensor
Simple mechanical problem - surrogate modelling
- Parametrised (patient-specific) mechanical problem
\[ \begin{cases} \boldsymbol{\nabla}\cdot \mathcal{\boldsymbol{\sigma}}+ \boldsymbol{f}\left(\left\{\mu_i\right\}_{i \in \mathopen{~[\!\![~}1, \beta \mathclose{~]\!\!]}}\right) = 0 \qquad & \mathrm{in} \ \Omega \\ \mathcal{\boldsymbol{\sigma}}\cdot \boldsymbol{n} = \boldsymbol{F}\left(\left\{\mu_i\right\}_{i \in \mathopen{~[\!\![~}1, \beta \mathclose{~]\!\!]}}\right) & \text{on } \partial \Omega_{N} \\ \boldsymbol{u} = \boldsymbol{u}_d \qquad \qquad & \text{on } \partial \Omega_{d} \\ \end{cases} \label{eq:MechPb2} \]
Behaviour
- \(\mathcal{\boldsymbol{\sigma}}= \mathbb{C}\left(\left\{\mu_i\right\}_{i \in \mathopen{~[\!\![~}1, \beta \mathclose{~]\!\!]}}\right):\mathcal{\boldsymbol{\varepsilon}}(\boldsymbol{u})\)
- \(\mathbb{C}\left(\left\{\mu_i\right\}_{i \in \mathopen{~[\!\![~}1, \beta \mathclose{~]\!\!]}}\right)\) the parametrised Hooke’s tensor
- \(\mathcal{\boldsymbol{\varepsilon}}(\boldsymbol{u})\) the linearised Green-Lagrange strain tensor
- \(\mathcal{\boldsymbol{\sigma}}\) the Cauchy stress tensor
Reduced-order model (ROM)
- Build a surrogate model
- Parametrised solution
- No computations online
- On the fly evaluation of the model
- Real-time parameters estimation
Interpolation of the solution
- FEM finite admissible space
\[ \mathcal{U}_h = \left\{\boldsymbol{u}_h \; | \; \boldsymbol{u}_h \in \text{Span}\left( \left\{ N_i^{\Omega}\left(\boldsymbol{x} \right)\right\}_{i \in \mathopen{~[\!\![~}1,N\mathclose{~]\!\!]}} \right)^d \text{, } \boldsymbol{u}_h = \boldsymbol{u}_d \text{ on }\partial \Omega_d \right\} \]
Interpretabilty Kronecker propriety Easy to prescribe Dirichlet boundary conditions
Fixed mesh
- Physics Informed Neural Networks (PINNs)
\[ \boldsymbol{u} \left(x_{0,0,0} \right) = \sum\limits_{i = 0}^C \sum\limits_{j = 0}^{N_i} \sigma \left( \sum\limits_{k = 0}^{M_{i,j}} b_{i,j}+\omega_{i,j,k}~ x_{i,j,k} \right) \]
All interpolation parameters are trainable Benefits from ML developments
Not easily interpretable Difficult to prescribe Dirichlet boundary conditions
- HiDeNN framework (Liu et al., 2023; Park et al., 2023; Zhang et al., 2021)
- Best of both worlds
- Reproducing FEM interpolation with constrained sparse neural networks
- \(N_i^{\Omega}\) are SNN with constrained weights and biases
- Fully interpretable parameters
- Continuous interpolated field that can be automatically differentiated
- Runs on GPUs
Solving the mechanical problem
Solving the mechanical problems amounts to finding the continuous displacement field minimising the potential energy \[E_p\left(\boldsymbol{u}\right) = \frac{1}{2} \int_{\Omega}\mathcal{\boldsymbol{\varepsilon}}: \mathbb{C} : \mathcal{\boldsymbol{\varepsilon}}~\mathrm{d}\Omega- \int_{\partial \Omega_N}\boldsymbol{F}\cdot \boldsymbol{u} ~\mathrm{d}A- \int_{\Omega}\boldsymbol{f}\cdot\boldsymbol{u}~\mathrm{d}\Omega. \]
In practice
- Compute the loss \(\mathcal{L} := E_p\)
- Find the parameters minimising the loss
- Rely on state-of-the-art optimizers (ADAM, LBFGS, etc.)
Degrees of freedom (Dofs)
- Nodal values (standard FEM Dofs)
- Nodal coordinates (Mesh adaptation)
- hr-adaptivity
| Dofs (initially) | \(569~438\) |
|---|---|
| Training time (s) | \(7\) |
| GPU (V100) | 1 |
| CPUs | 40 |
More details:
Katerina Skardova at 4pm tomorrow in 02.011
Proper Generalised Decomposition (PGD)
PGD: (Chinesta et al., 2011; Ladevèze, 1985)
Tensor decomposition
- Separation of variables
- Low-rank \(m\)
- \(\textcolor{BleuLMPS}{\overline{\boldsymbol{u}}_i(\boldsymbol{x})}\) Space modes
- \(\textcolor{LGreenLMS}{\lambda_i^j(\mu^j)}\) Parameter modes
\[\textcolor{VioletLMS}{\boldsymbol{u}}\left(\textcolor{BleuLMPS}{\boldsymbol{x}}, \textcolor{LGreenLMS}{\mu}\right) = \sum\limits_{i=0}^{m}\textcolor{BleuLMPS}{\overline{u}_i\left(x\right)}\textcolor{LGreenLMS}{\lambda_i\left(\mu\right)}\]
Discretised problem
- From \(N\times N_{\mu}\) unknowns to \(m\times\left(N + N_{\mu}\right)\)
Proper Generalised Decomposition (PGD)
PGD: (Chinesta et al., 2011; Ladevèze, 1985)
Tensor decomposition
- Separation of variables
- Low-rank \(m\)
- \(\textcolor{BleuLMPS}{\overline{\boldsymbol{u}}_i(\boldsymbol{x})}\) Space modes
- \(\textcolor{LGreenLMS}{\lambda_i^j(\mu^j)}\) Parameter modes
\[ \textcolor{VioletLMS}{\boldsymbol{u}}\left(\textcolor{BleuLMPS}{\boldsymbol{x}}, \textcolor{LGreenLMS}{\left\{\mu_i\right\}_{i \in \mathopen{~[\!\![~}1, \beta \mathclose{~]\!\!]}}}\right) = \sum\limits_{i=1}^m \textcolor{BleuLMPS}{\overline{\boldsymbol{u}}_i(\boldsymbol{x})} ~\textcolor{LGreenLMS}{\prod_{j=1}^{\beta}\lambda_i^j(\mu^j)} \]
Discretised problem
- From \(N\times\prod\limits_{j=1}^{~\beta} N_{\mu}^j\) unknowns to \(m\times\left(N + \sum\limits_{j=1}^{\beta} N_{\mu}^j\right)\)
- Finding the tensor decomposition by minimising the strain energy \[ \left(\left\{\overline{\boldsymbol{u}}_i \right\}_{i\in \mathopen{~[\!\![~}1,m\mathclose{~]\!\!]}},\left\{\lambda_i^j \right\}_{ \begin{cases} i\in \mathopen{~[\!\![~}1,m\mathclose{~]\!\!]}\\ j\in \mathopen{~[\!\![~}1,\beta \mathclose{~]\!\!]} \end{cases} } \right) = \mathop{\mathrm{arg\,min}}_{ \begin{cases} \left(\overline{\boldsymbol{u}}_1, \left\{\overline{\boldsymbol{u}}_i \right\} \right) & \in \mathcal{U}\times \mathcal{U}_0 \\ \left\{\left\{\lambda_i^j \right\}\right\} & \in \left( \bigtimes_{j=1}^{~\beta} \mathcal{L}_2\left(\mathcal{B}_j\right) \right)^{m-1} \end{cases}} ~\underbrace{\int_{\mathcal{B}}\left[E_p\left(\boldsymbol{u}\left(\boldsymbol{x},\left\{\mu_i\right\}_{i \in \mathopen{~[\!\![~}1, \beta \mathclose{~]\!\!]}}\right), \mathbb{C}, \boldsymbol{F}, \boldsymbol{f} \right) \right]\mathrm{d}\beta}_{\mathcal{L}} \label{eq:min_problem} \]
Neural Network PGD
Graphical implementation of Neural Network PGD
\[ \boldsymbol{u}\left(\textcolor{BleuLMPS}{\boldsymbol{x}}, \textcolor{LGreenLMS}{\left\{\mu_i\right\}_{i \in \mathopen{~[\!\![~}1, \beta \mathclose{~]\!\!]}}}\right) = \sum\limits_{i=1}^m \textcolor{BleuLMPS}{\overline{\boldsymbol{u}}_i(\boldsymbol{x})} ~\textcolor{LGreenLMS}{\prod_{j=1}^{\beta}\lambda_i^j(\mu^j)} \]
Interpretable NN-PGD
- No black box
- Fully interpretable implementation
- Great transfer learning capabilities
- Straightforward implementation
- Benefiting from current ML developments
- Straightforward definition of the physical loss using auto-differentiation
Stiffness and external forces parametrisation
Illustration of the surrogate model in use
Parameters
- Parametrised stiffness \(E\)
- Parametrised external force \(\boldsymbol{f} = \begin{bmatrix} ~ \rho g \sin\left( \theta \right) \\ -\rho g \cos\left( \theta \right) \end{bmatrix}\)
NN-PGD
- Immediate evaluation of the surrogate model
- Straightforward differentiation capabilities regarding the input parameters
Multi-level training
Strategy
- The interpretability of the NN allows
- Great transfer learning capabilities
- Start with coarse (cheap) training
- Followed by fine tuning, refining all modes
- Great transfer learning capabilities
- Extends the idea put forward by (Giacoma et al., 2015)
Note
- The structure of the Tensor decomposition is kept throughout the multi-level training
- Last mode of each level is irrelevant by construction
- It is removed before passing onto the next refinement level
Neural Network PGD
- Bi-stiffness structure with unknown
- Stiffness jump \(\Delta E\)
- Jump position \(\alpha\)
- Greed strategy (training) of the surrogate model
- \(9\) PGD modes
- Mode addition when stagnation
Conclusion
Conclusion
- Robust and straightforward general implementation of a NN-PGD
- Interpretable
- Benefits from all recent developments in machine learning
- Surrogate modelling of parametrised PDE solution
- Promising results on simple linear toy case
Perspectives for patient-specific applications
- Implementation of the poro-mechanics lung model (Patte et al., 2022)
- Parametrisation of the segmented geometries
- Variational encoder to reduce the segmentation’s parametrisation’s dimensionnality
- Application to real-time lung parameters estimation
- Error quantification on the estimated parameters
Technical perspectives
- 3D implementation
- Publication of the code …
- … to come shortly
- Do not hesitate to check the Github page
- … to come shortly
References
