Best Network Architecture

[engsbk]

Hello everyone!

I'm trying to model highly oscillatory functions ( like the 2D wave function with a source point) via using FNNs, but I can only get limited accuracy despite using numerous approaches:

• LAAF
• observation points
• mini-batch
• resampling
• large epoch number
• large networks

Is there any other type of network in deepxde that performed better in your case than FNN? Please share and let me know. I'm looking forward to your responses. Sharing your code implementation of the network if it showed good results is appreciated.

[lululxvi]

Maybe the following could be helpful

• https://deepxde.readthedocs.io/en/latest/demos/pinn_forward/poisson.1d.multiscaleFourier.html
• https://github.com/lululxvi/deepxde/blob/master/examples/pinn_forward/wave_1d.py

[praksharma]

Solving Waves equations is hard. Even Nvidia Modulus experiences lots of problems. Recently, they have implemented FNOs (Fourier neural operator) to solve Wave equations. Have a look at it. I am also playing with Nvidia Modulus for a long time.

[lululxvi]

Here is an example of wave equation: https://github.com/lululxvi/deepxde/blob/master/examples/pinn_forward/wave_1d.py

[engsbk]

Here is an example of wave equation: https://github.com/lululxvi/deepxde/blob/master/examples/pinn_forward/wave_1d.py

In this example, I have a couple of questions:

1. The L2 relative error is used as a loss term, correct? what if I don't have a solution to compare with, and I removed the L2 loss error, will it still produce good results?

2. I noticed also that ic_1 uses func() wich contains a variable t. Wouldn't this apply only when t is equal to zero since it is an initial condition?

3. I also noticed that the sigma values sigmas_t=[1, 10] are set to these values specifically. My guess is that is because A and C are on these order, if not then how to choose the sigma values?

4. If I'm expanding this problem for 2D, how does that affect the values of sigma?

Looking forward to your reply!

[praksharma]

Sorry for the interruption.

1. The true solution does not participate in the training. It just computes the test error to see the performance of the network. I do agree that extracting the boundary conditions and initial conditions from the solution function creates confusion. But try to understand, that the BC and IC are part of the Spatio-temporal solution, so it doesn't matter if you extract it from the solution function or define it yourself.
2. In this line ic_1 = dde.icbc.IC(geomtime, func, lambda _, on_initial: on_initial), you see that lambda function? So, if you see this commit from Apr 2021. All the inputs were vectorized and special strings were used to denote the special entities such as on_boundary, on_initial. So if you go here, You will see this

class TimeDomain(Interval):
    def __init__(self, t0, t1):
        super().__init__(t0, t1)
        self.t0 = t0
        self.t1 = t1

    def on_initial(self, t):
        return np.isclose(t, self.t0).flatten()

where t0 is the initial time and t1 is the final time. I hope it is clear.

3. I would encourage you to see the paper on FNN. Here you will find the code that DeepXDE implements and before the __init__() function you will see the link to the paper it refers to for each class.
4. Again, read the paper. Also, look at more advanced techniques such as AFNOs (adaptive Fourier neural operator).

[lululxvi]

4. Again, read the paper. Also, look at more advanced techniques such as [AFNOs (adaptive Fourier neural operator)](https://arxiv.org/abs/2111.13587).

One more comment, AFNO (or FNO, or DeepONet) is developed with a different purpose (in some cases they can replace PINN). See some discussion at here

[praksharma]

@lululxvi Thanks a lot for the paper. will go through it. Meanwhile, Nvidia launched the new Modulus 22.03, last Friday. They are using FNOs. I am just checking how suitable it is to solve PDEs.