NeuralPDE.jl icon indicating copy to clipboard operation
NeuralPDE.jl copied to clipboard

Add NeuralTangentKernel Loss

Open Parvfect opened this issue 2 years ago • 5 comments

Adapted from paper - https://arxiv.org/pdf/2007.14527.pdf Treating Issue #501

Issues

  1. Still not completely sure if values used in struct are to be calculated in the struct or somewhere else
  2. Relies on ForwardDiff.compute_jacobian
  3. Tests and dimension check
  4. Not completely clear on how reweighing takes place

Parvfect avatar Apr 05 '22 08:04 Parvfect

Relies on ForwardDiff.compute_jacobian

That's fine for now. The literature doesn't sem to say how to optimize this more, even though it's quite obvious it can be improved. Seems like a good paper idea, but for now just do it by the book.

Not completely clear on how reweighing takes place

Feel free to ask questions on the Slack! Zoe should be around?

Tests and dimension check

This is essential. See the other adaptive reweighing tests

ChrisRackauckas avatar Apr 05 '22 12:04 ChrisRackauckas

Haven't been able to get help from Zoe, I'll try Slack again later.

I'll go through the tests that are written already for other adaptive losses. Is it worth reproducing results from the paper?

Parvfect avatar Apr 05 '22 16:04 Parvfect

I'll go through the tests that are written already for other adaptive losses. Is it worth reproducing results from the paper?

It would be good to try. While it might be too expensive to be a unit test, it would make a good tutorial.

ChrisRackauckas avatar Apr 06 '22 00:04 ChrisRackauckas

I'm working on a project deadline for this Tuesday. I'll go through and add more detailed comments after that.

zoemcc avatar Apr 29 '22 03:04 zoemcc

This should get rebased due to changes in https://github.com/SciML/NeuralPDE.jl/pull/553. That would hopefully make it much cleaner too.

ChrisRackauckas avatar Jun 29 '22 19:06 ChrisRackauckas