PINN Test overhaul/expansion

[zoemcc]

I'm working on a systematic expansion and review of the tests for PINNs:

• [ ] Many PINN tests compare to the analytic solution but only using a test like: @test u_predict ≈ u_real atol = 3.0 with a magic number for atol. This should be replaced by two relative comparisons: average % error and maximum % error. It is difficult to tell where these atol numbers came from and if they're testing if the learned solution is sufficiently close. (example)
• [ ] Asymptotic Strategy Equivalence: I'm adding a series of tests that should test that each different sampling strategy approximately converges to the same loss function as the number of samples increases to infinity. I think this is a very desirable property for us to have as we can more directly compare losses across strategies. They currently do not, and so some tweaks will be required to the loss function generations (likely the tau constants that we pass in) to satisfy this property.
• [ ] Analytic solution tests: I'm adding a series of tests that give as input to the PINN the analytical solution of the PDE wrapped in a fastchain with 0 parameters. Then I test that the different strategies produce loss functions that are approximately 0 on the analytic solution that I give it (basically, the analytical solution should be a solution to our optimization problem). Relatedly, I want to perturb the analytical solution and verify that the perturbed version no longer has 0 loss.