JacquesCarette
Add NFR for Accuracy to SWHS
PR #4376 points out that Relative tolerance and Absolute tolerance only appear in the table of inputs. We need these numbers for the code generation, since the ODE solver needs a stopping criterion. If we had a design document between the requirements and the code, we could introduce this information there, but we don't currently have any document filling that role.
The nonfunctional requirement for accuracy could be something like:
For cases that can be tested, the computed solutions for $T_W(t)$ and $E_W(t)$ should agree with the true solution ($T_W(t)^\text{true}$ and $E_W(t)^\text{true}$) to satisfy the following equations:
$$(\forall t: \mathbb{R} | 0 \leq t \leq t_\text{final} : | T_W(t)^\text{true} - T_W(t) | \leq A_\text{tol} ) $$
$$(\forall t: \mathbb{R} | 0 \leq t \leq t_\text{final} : | E_W(t)^\text{true} - E_W(t) | \leq A_\text{tol} ) $$
$$(\forall t: \mathbb{R} | 0 \leq t \leq t_\text{final} : \left | \frac{T_W(t)^\text{true} - T_W(t)} { T_W(t)^\text{true}} \right | \leq R_\text{tol} ) $$
$$(\forall t: \mathbb{R} | 0 \leq t \leq t_\text{final} : \left | \frac{E_W(t)^\text{true} - E_W(t)} { E_W(t)^\text{true}} \right | \leq R_\text{tol} ) $$
This issue brings up some other issues. It really isn't feasible to have an NFR that puts a bound on the accuracy of the solution. In general, we can't prove that the numerical solution is always going to be within those bounds. I added some wiggle words above to say "for cases that can be tested". What we really should do is have a verification and validation plan that is based on assurance case arguments for what is adequate testing. We don't currently have this, but at least adding in a requirement will remind us that we need more detail.
The equations given above for absolute and relative accuracy could become part of a theory of accuracy, which could then be shared between many of our examples, since the notion of accuracy comes up all the time in research software.
We might want to have a "proto design" document that would (currently) be completely unstructured, but would at least display the extra knowledge that we need. Of course, we don't need such a document for the knowledge to exist in our 'database' of information for SWHS.
Theoretical issues of doing forward error or backward error, verification, etc, are somewhat orthogonal to recording what we want. If we want 'stupid things', well, that is another matter!
@JacquesCarette I agree with the "proto design" document.
I'm not sure I agree that verification is a stupid thing to do? Maybe I misunderstood your point? If you mean this content isn't part of the SRS document, I agree it isn't part of the SRS. Verification should be part of the process, and I imagine that someday we'll generate documentation related to verification.
Attempting to verify an impossible requirement is... stupid. Verification is great. Unachievable requirements are bad.