NeuralPDE.jl
NeuralPDE.jl copied to clipboard
Add NeuralTangentKernel Loss
Adapted from paper - https://arxiv.org/pdf/2007.14527.pdf Treating Issue #501
Issues
- Still not completely sure if values used in struct are to be calculated in the struct or somewhere else
- Relies on ForwardDiff.compute_jacobian
- Tests and dimension check
- Not completely clear on how reweighing takes place
Relies on ForwardDiff.compute_jacobian
That's fine for now. The literature doesn't sem to say how to optimize this more, even though it's quite obvious it can be improved. Seems like a good paper idea, but for now just do it by the book.
Not completely clear on how reweighing takes place
Feel free to ask questions on the Slack! Zoe should be around?
Tests and dimension check
This is essential. See the other adaptive reweighing tests
Haven't been able to get help from Zoe, I'll try Slack again later.
I'll go through the tests that are written already for other adaptive losses. Is it worth reproducing results from the paper?
I'll go through the tests that are written already for other adaptive losses. Is it worth reproducing results from the paper?
It would be good to try. While it might be too expensive to be a unit test, it would make a good tutorial.
I'm working on a project deadline for this Tuesday. I'll go through and add more detailed comments after that.
This should get rebased due to changes in https://github.com/SciML/NeuralPDE.jl/pull/553. That would hopefully make it much cleaner too.