Exploring Neural Nets 
for PDE’s

Ricardo López

Outline

Why use Neural Networks (NN’s) for PDE’s?

What is a Neural Network?

What is a Physics Informed Neural Network (PINN)?

Theoretical Approaches: High dimension PDE’s

Deep BSDE (Backward Stochastic Differential Equation) method

Variational Quantum MonteCarlo Method

Why consider NN’s for solving PDE’s?

Traditional methods to find approximate solutions of PDE’s (Numerically)

({{Traditional numerical approach, Neural Network Approach}, {FEM (finite element method)
FDM (finite difference method), Neural Networks(BSDE's, 
Variational Quantum Montecarlo...)}, {Spatial discretization, Do not rely on a grid}, {Curse of Dimensionality (CoD), Potentially free of CoD}, {Some solutions demand 
tailored approaches, Efficient evaluating new data points 
(once trained)}})

What is a Neural Network?

Neuron: basic unit of a 
NN: An equation with a bunch of parameters

Neuron

Neural Network

What is a Neural Network?

y==f((Subscript[w, 1]) (Subscript[x, 1])+(Subscript[w, 2]) (Subscript[x, 2])+(Subscript[w, 3]) (Subscript[x, 3])+b)

Parts

y: output

(Subscript[x, n]): input

(Subscript[w, n]): weight of a connection

ϝ: activation function

b: bias

Structure of a NN

Training the NN

Adjust weights and biases using training data

Cost function: minimize

What is a Neural Network?

->->

What is a Physics Informed Neural Network? (PINN)

Many physical phenomenons are modeled by PDE’s

PINN’s is a type of NN that integrates physical laws into the training process of the NN

How is it done?

e.g finding solutions of damped harmonic oscillator PDE

The term added to the cost function tries to ensure that the solution given by the NN is consistent with the known physics

({{Classical NN, Classical NN, PINN}, {Schematic, , }, {Cost function: training, (1/N)(∑_(i=1)^n)((((Subscript[u, NN])((Subscript[x, i]),θ)-(Subscript[u, true])((Subscript[x, i]))))^2), (1/N)(∑_(i=1)^n)((((Subscript[u, NN])((Subscript[x, i]),θ)-(Subscript[u, true])((Subscript[x, i]))))^2)+
(1/M)(∑_(j=1)^M)((([((m (d^2))/(dx^2))+((µd)/dx)+κ](Subscript[u, NN])((Subscript[x, j]),θ)))^2)}, {Training, , }})

Neural Network

Import["C:\\Users\\ASUS\\Downloads\\nn.gif","AnimatedImage"]

PINN

Import["C:\\Users\\ASUS\\Downloads\\pinn-1.gif","AnimatedImage"]

Burgers’ equation:

Wolfram language for PINNs:

https://community.wolfram.com/groups/-/m/t/2446710

The Deep BSDE method

BSDE stands for Backward Stochastic Differential Equation

First deep learning based algorithm for solving non-linear parabolic PDEs

Does not suffer from the curse of dimensionality

The Deep BSDE method

Semi-linear parabolic PDE:

Parts

x: d-dimensional space variable

μ : vector valued function

σ: d × d matrix valued function

(Subscript[Hess, x]) : Hessian matrix

f : known non linear function

The Deep BSDE method

If (Subscript[W, s]) is a Brownian motion, and (Subscript[X, t]):

Then:

The Deep BSDE method

Approximate the integrals:

And:

Where:

The Deep BSDE method

Using neural networks:

By defining

GraphicsGrid[{{, }}]

The Deep BSDE method

Using neural networks:

Parameters and cost function

Variational Quantum MonteCarlo Method

Multi-particle Schrödinger equation

Variational Quantum MonteCarlo Method

Variational principle

Thanks