Adaptive activation function

[LanPeng-94]

Hi Dr.Lu I found a discussion of adaptive activation functions in the open issues, and you mentioned that the "https://deepxde.readthedocs.io/en/latest/modules/deepxde.maps.html#deepxde.maps.activations.layer_wise_locally_adaptive" link has a related report. Unfortunately, it seems to be disabled, where can I go to find it?

[lululxvi]

https://deepxde.readthedocs.io/en/latest/modules/deepxde.nn.html#deepxde.nn.activations.layer_wise_locally_adaptive

[LanPeng-94]

Thank you for your answer. I have one more question, is Deepxde able to implement "Parallel physics-informed neural networks via domain decomposition (References:https://www.sciencedirect.com/science/article/pii/S0021999121005787 )", if so, can you tell me how to do it?

[lululxvi]

• Domain decomposition is doable by modifying the code.
• implementing parallel-GPU PINN is doable, but it requires some significant effort, because currently DeepXDE only supports the training in one GPU.

[LanPeng-94]

• Domain decomposition is doable by modifying the code.

• Thank you for your answer. For domain decomposition, my idea is the same as PFNN to build pairs of neural networks to model different regions of the solution separately. For example, for 2 regions, assuming that the region junction is x=xI, we construct NN_U and NN_L respectively, so that we can define the loss function NN_U(xI,ti)-NN_L(xI,ti) and NN_Ux(xI,ti)-NN_Lx(xI,ti). How should we go about defining these two loss functions

• I was recently trying gPINN. I was solving the Burgers equation and I thought that using gPINN was going to add a PDE loss, as in the following code

def pde(x, y):
    dy_x = dde.grad.jacobian(y, x, i=0, j=0)
    dy_t = dde.grad.jacobian(y, x, i=0, j=1)
    dy_xx = dde.grad.hessian(y, x, i=0, j=0)
    return dy_t + y * dy_x - 0.01 / np.pi * dy_xx

Modified to

def pde(x, y):
    dy_x = dde.grad.jacobian(y, x, i=0, j=0)
    dy_t = dde.grad.jacobian(y, x, i=0, j=1)
    dy_xx = dde.grad.hessian(y, x, i=0, j=0)
    dy_xt = dde.grad.hessian(y, x, i=0, j=1)
    dy_xxx = dde.grad.jacobian(dy_xx, x, i=0, j=0)
    eq_a = dy_t + y * dy_x - 0.01 / np.pi * dy_xx
    eq_ax = dy_xt+( dy_x* dy_x+y* dy_xx)-0.01 / np.pi * dy_xxx
    return [eq_a, eq_ax]

Is this the idea？

[lululxvi]

Both are doable.

• Simply consider NN_U and NN_L are two functions defined on the entire domain.
• See gPINN code at https://github.com/lu-group/gpinn

[ShaikhaTheGreen]

How to merge the results of NN_U and NN_L together?

Thanks for the great question @979736316, if it worked for you, please share the code.

I’m trying to improve my results with no luck until now.

[lululxvi]

See FAQ "Q: How can I implement new ODEs/PDEs, e.g., compute derivatives, complicated PDEs?". There are some examples there.

[ShaikhaTheGreen]

I haven't found an example similar to what I'm trying to do. I'm trying to model the 2D-space 1D-time wave equation as follows

f(x,5,t)=f(0,y,t)=f(5,y,t)=f(x,0,t)=0, where all the boundries are absorbing (zero) a source function f(0.5,2.5,t)= 2 * sin (2 pi t) and an initial condition f(x,y,0) = 0

The problem is with having the source function modeled accurately. The best result I got so far is L2 error = 1.29:

Any assistance is very much appreciated.

[LanPeng-94]

How to merge the results of NN_U and NN_L together?

Thanks for the great question @979736316, if it worked for you, please share the code.

I’m trying to improve my results with no luck until now.

I'm sorry I forgot to reply to you. I haven't implemented this idea yet, but have some idea that it might be feasible to use OperatorBC to define the boundary at the intersection x=I.

[gaboperezayala]

Thanks again for this question @979736316, I have been facing the same issue, after implementing the OperatorBC I am getting the same error as #299 (TypeError: 'function' object is not subscriptable) , still dont know how to solve the problem, I'd really appreciate any help!!

[lululxvi]

@engsbk You try to model a point source? If so, this is not a problem of implementation in DeepXDE, instead, it is a problem of the current PINN method.

[railleon]

@lululxvi Dear Dr Lu. Thank you very much for the excellent tools you have developed, I am also facing the same problem of point sources. I would like to know why DeepXDE cannot achieve the simulation of point sources?

Currently, I am using the function of the boundary to input the excitation function of the point source and the multiple points which are very close to it, so as to realize the simulation of the point source, is it feasible?

[lululxvi]

A point source is essentially a delta function. For a point source, roughly all the numerical methods cannot handle point source directly. If you consider FEM, a point source is essentially modeled as a source in that mesh. Similarly for PINN, you need to approximate the point source by a source in some (small) area. In physics, you would also not be able to have an exact point source.

Of course, you can do it in PINN to implement a point source, but I am not sure about the accuracy.

[ShaikhaTheGreen]

@engsbk You try to model a point source? If so, this is not a problem of implementation in DeepXDE, instead, it is a problem of the current PINN method.

I still think that the accuracy can be improved somehow. I thought about using anchor points or point sets but the accuracy can only be in the region of the number I mentioned earlier.

Is there a way to consider the same point in space across different time instances throughout all the training epochs? I’m not sure if this is what anchor points already does. Just thinking out loud here.. Let me know of any suggestions or ideas you have to solve this problem.

[lululxvi]

Yes, you can use anchors to define any point you like.

[yuzew]

I haven't found an example similar to what I'm trying to do. I'm trying to model the 2D-space 1D-time wave equation as follows

f(x,5,t)=f(0,y,t)=f(5,y,t)=f(x,0,t)=0, where all the boundries are absorbing (zero) a source function f(0.5,2.5,t)= 2 * sin (2 pi t) and an initial condition f(x,y,0) = 0

The problem is with having the source function modeled accurately. The best result I got so far is L2 error = 1.29:

Any assistance is very much appreciated.

Dear @engsbk do you still work on the point source 2D wave equation? If you still do that, could you please share the code? I will really appreciate it.

[sai-karthikeya-vemuri]

Dear @engsbk ,

I am really impressed by your work on this topic !

I am also learning to model wave equations and running into many problems .

It would be much helpful if you could share the code for 2d wave equation .

Cheers, Sai

[ShaikhaTheGreen]

Hello @yuzew @sai-karthikeya-vemuri I’m sorry for the late reply. This project is still in the works, so we have not yet published the code. Is there any particular question/part of the model I can help you with?

[sai-karthikeya-vemuri]

Dear @engsbk ,

Thanks for the reply!

Since normal PINNs failed , I just wanted to know which strategy works better.

Regarding that:

Is it better to use baseline PINN or gPINN or MultiScalePINN ?
Did you use any loss weighing schemes or hard BCs?
Does residual adaptive resampling strategies improve the solution (which one is recommended in the case)?

I am using 5000 points inside domain and 1000 for BC and IC

Any type of suggestion on how to get good result like you would be appreciated!

I also saw that you tried for 1d case , can you provide the code snippet for 1d case?

With best regards, Sai

[Lujiang-ECUST]

您好，很抱歉回复晚了。这个项目仍在进行中，所以我们还没有发布代码。有什么特别的问题/模型的一部分可以帮助你吗？ Dear @engsbk ,Hello, I am very inspired by the work you have discussed, could you provide the code for this part of the work , it will help me a lot!