clad icon indicating copy to clipboard operation
clad copied to clipboard

Add ability to print FP errors in error estimation.

Open grimmmyshini opened this issue 2 years ago • 11 comments

Allow users to print intermediate floating-point errors to a file. This PR will allow users to use the "-fprint-error-ests-to-file" and enable printing of whatever they specify in the custom model's Print function. This will not only enable users to get access to intermediate error and derivative values (something they would not have otherwise) automatically but also allow them a deeper insight into the error analysis. One can perform analysis like error sensitivity of variables through this feature, all they have to do is implement the Print function as such it prints the sensitivity of each variable of interest.

A possible pitfall of this implementation is that we have no way (at least not a non-complex way) to attach source information to the "Print" values of each variable, i.e. currently we print to the file something like this: "var_name : <--expr specified by Print-->". Now, this is very terse and can easily become confusing for larger codes where the same variable appears multiple times as the variable of interest. A better implementation would see the printing of something like: "var_name : var_src_line_no : <--expr specified by Print-->" where the var_src_line_no specifies the line number of the occurrence of that specific variable, tying each "Print" value to an expression in the code.

  • [ ] ~Add a demo for sensitivity analysis. (closes #295)~
  • [ ] Update the website tutorial to add the Print function there.

grimmmyshini avatar Feb 02 '22 19:02 grimmmyshini

Once I rebase against #304, I will switch the SetErrorObject() method to use clad::utils::CreateStringLiteral instead of ASTContext::getStringLiteralArrayType (which probably is not present everywhere).

grimmmyshini avatar Feb 02 '22 19:02 grimmmyshini

Codecov Report

Merging #370 (bded35f) into master (2d5e578) will decrease coverage by 0.26%. The diff coverage is 97.05%.

:exclamation: Current head bded35f differs from pull request most recent head 0b83295. Consider uploading reports for the commit 0b83295 to get more accurate results

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #370      +/-   ##
==========================================
- Coverage   92.52%   92.26%   -0.27%     
==========================================
  Files          35       35              
  Lines        5364     5274      -90     
==========================================
- Hits         4963     4866      -97     
- Misses        401      408       +7     
Impacted Files Coverage Δ
demos/ErrorEstimation/CustomModel/CustomModel.h 100.00% <ø> (ø)
include/clad/Differentiator/DerivativeBuilder.h 100.00% <ø> (ø)
include/clad/Differentiator/DiffPlanner.h 100.00% <ø> (ø)
include/clad/Differentiator/EstimationModel.h 90.90% <0.00%> (-9.10%) :arrow_down:
lib/Differentiator/ReverseModeVisitor.cpp 95.51% <ø> (-0.35%) :arrow_down:
lib/Differentiator/ErrorEstimator.cpp 98.69% <97.26%> (-0.42%) :arrow_down:
include/clad/Differentiator/ErrorEstimator.h 100.00% <100.00%> (ø)
lib/Differentiator/DerivativeBuilder.cpp 100.00% <100.00%> (ø)
lib/Differentiator/DiffPlanner.cpp 99.64% <100.00%> (-0.03%) :arrow_down:
lib/Differentiator/EstimationModel.cpp 100.00% <100.00%> (ø)
... and 18 more
Impacted Files Coverage Δ
demos/ErrorEstimation/CustomModel/CustomModel.h 100.00% <ø> (ø)
include/clad/Differentiator/DerivativeBuilder.h 100.00% <ø> (ø)
include/clad/Differentiator/DiffPlanner.h 100.00% <ø> (ø)
include/clad/Differentiator/EstimationModel.h 90.90% <0.00%> (-9.10%) :arrow_down:
lib/Differentiator/ReverseModeVisitor.cpp 95.51% <ø> (-0.35%) :arrow_down:
lib/Differentiator/ErrorEstimator.cpp 98.69% <97.26%> (-0.42%) :arrow_down:
include/clad/Differentiator/ErrorEstimator.h 100.00% <100.00%> (ø)
lib/Differentiator/DerivativeBuilder.cpp 100.00% <100.00%> (ø)
lib/Differentiator/DiffPlanner.cpp 99.64% <100.00%> (-0.03%) :arrow_down:
lib/Differentiator/EstimationModel.cpp 100.00% <100.00%> (ø)
... and 18 more

codecov[bot] avatar Mar 08 '22 12:03 codecov[bot]

I will squash and format the changes once the changes get a green light.

grimmmyshini avatar May 03 '22 12:05 grimmmyshini

@vgvassilev Can you greenlight the changes before I squash and format? Or would you rather review after I do those things? Because it is very difficult to rebase over multiple related commits.

grimmmyshini avatar Jun 08 '22 21:06 grimmmyshini

I had some suggestions but this is good to go. Please squash and format.

vgvassilev avatar Jun 09 '22 05:06 vgvassilev

@vgvassilev, we have to pause this, test failures due to #459

grimmmyshini avatar Jun 10 '22 10:06 grimmmyshini

As per @parth-07's comments, added the same workaround for #459

grimmmyshini avatar Jun 13 '22 14:06 grimmmyshini

As per @parth-07's comments, added the same workaround for #459

Which workaround did you use? And can you please briefly explain why was any workaround required for error estimation here?

parth-07 avatar Jun 13 '22 19:06 parth-07

As per @parth-07's comments, added the same workaround for #459

Which workaround did you use? And can you please briefly explain why was any workaround required for error estimation here?

By workaround I mean using what is done everywhere else, i.e. OutputParamType_t<Args, void>.

grimmmyshini avatar Jun 14 '22 08:06 grimmmyshini

Which workaround did you use? And can you please briefly explain why was any workaround required for error estimation here?

By workaround I mean using what is done everywhere else, i.e. OutputParamType_t<Args, void>.

Oh okay, now I understand. Thanks!

parth-07 avatar Jun 14 '22 08:06 parth-07

@grimmmyshini apologies for the delay, can you rebase?

vgvassilev avatar Jun 23 '22 18:06 vgvassilev

I was wondering what is the fate of this PR?

vgvassilev avatar Dec 07 '22 21:12 vgvassilev

I would recommend closing this in favor of the demo print model added by #512. That way to print errors is more flexible.

grimmmyshini avatar Dec 08 '22 01:12 grimmmyshini

Is #512 strictly better?

vgvassilev avatar Dec 09 '22 07:12 vgvassilev

I don't know if it's strictly better but i don't see any reason for not using the print model... This version is very rigid.

grimmmyshini avatar Dec 09 '22 11:12 grimmmyshini

Okay, then please close this PR.

vgvassilev avatar Dec 09 '22 11:12 vgvassilev