modelica
Enriched tool-independent experiment annotation
There are a few settings stored in an annotation common to all tools: StartTime, StopTime, Interval, Tolerance. I think it's worth to discuss which settings shouldn't be stored in a tools specific annotation but in a common one, e.g. Output(Protected variables, Equidistant time grid, Store variables at events, ...)
I can understand that feature, but there's also a need to innovate and replace them with something better:
- Instead of just storing (or not storing) protected variables tools (such as Dymola) may allow you to select (or filter) variables in more advanced ways. I don't know if we could propose that
- To me not storing at events is an indication that you have something special causing "events" (such as discretized variables for a clock) where some more specialized solution would be better suited.
- In the linked issue the problem is how to capture transients and normal simulations in an efficient way. To me "non-equidistant grid" is not ideal for that, but kind of works. One possibility (implemented in Dymola) is to manually modify the grid rate during the simulation, but there might be more automatic ways.
I don't know what other tools do for these cases.
- To me "non-equidistant grid" is not ideal for that, but kind of works.
I'd be interested in what aspect of this you feel is not ideal, could you maybe elaborate? AFAICT/in my experience, this results in the "grid rate" as dense as the solver needs to observe the required tolerance. Isn't this the "more [actually, fully] automatic way" you are thinking about? It gives you detail where the simulation requires it because "interesting" things happen quickly, and takes larger steps where the transients are slow?
- To me "non-equidistant grid" is not ideal for that, but kind of works.
I'd be interested in what aspect of this you feel is not ideal, could you maybe elaborate? AFAICT/in my experience, this results in the "grid rate" as dense as the solver needs to observe the required tolerance.
There are some reason I find it non-ideal: The solver uses a higher order interpolation - the solution does not (well, depending on tool) so storing once per step is a bit crude and this makes it messy when comparing different solvers. This is especially noticeable for periodic signals, especially after the transient.
I can understand that feature, but there's also a need to innovate and replace them with something better:
* Instead of just storing (or not storing) protected variables tools (such as Dymola) may allow you to select (or filter) variables in more advanced ways. I don't know if we could propose that
OpenModelica also allows that, by using regexp filters on variable names. But that's a different feature.
I only know Dymola and OpenModelica; both have a checkbox (default: unchecked) for storing protected variables. People have been happily using this feature for years. The only problem is, it's not standardized, so when you switch tool, it doesn't work. Why don't we just start standardizing stuff that is well-established? This doesn not preclude doing something fancier later.
* To me not storing at events is an indication that you have something special causing "events" (such as discretized variables for a clock) where some more specialized solution would be better suited.
For some users (e.g. @luizmac in OpenModelica/OpenModelica#5709) the requirement is to get periodically sampled outputs, to which you can, e.g., apply FFT analysis or discrete-time system identification techniques. In all such cases, the requirement is to have exactly one data point after every Interval of time, and nothing else.
This requirement is absolutely reasonable and I believe it would deserve an experiment annotation.
* In the linked issue the problem is how to capture transients and normal simulations in an efficient way. To me "non-equidistant grid" is not ideal for that, but kind of works. One possibility (implemented in Dymola) is to manually modify the grid rate during the simulation, but there might be more automatic ways.I don't know what other tools do for these cases.
OpenModelica at the moment is providing what solvers like DASSL or IDA already provide, i.e., either they internally advance the simulation with a variable step size solver and return periodically sampled solutions via dense output (what has been informally called "Equidistant time grid" so far, or they just return the solution at the steps taken by the solver, which is what you get if you switch off "Equidistant time grid". I agree with @HansOlsson that "No Equidistant time grid" kind of works when systems have fast transients every now and then, because you get more tight time points where you need them. But in some cases it gives serious aliasing problems, see the discussion in this ticket.
One option that has been put forth is to output the solution both at regular intervals and at variable time steps, though I understand this feature is not available natively in solvers such as IDA.
One option that has been put forth is to output the solution both at regular intervals and at variable time steps, though I understand this feature is not available natively in solvers such as IDA.
With "natively", am I guessing correctly that you mean "both accessible from one unchanged function call" or somesuch?
If not, AFAICT this is possible e.g. in CVODE, but you need some logic in your solver-calling code to e.g. do internal solver steps with CV_ONE_STEP, and if regular output times have been stepped over, interpolate those results from the solver output at high accuracy.
I suspect the API for IDA will be equivalent.
I can understand that feature, but there's also a need to innovate and replace them with something better:
* Instead of just storing (or not storing) protected variables tools (such as Dymola) may allow you to select (or filter) variables in more advanced ways. I don't know if we could propose thatOpenModelica also allows that, by using regexp filters on variable names. But that's a different feature.
I only know Dymola and OpenModelica; both have a checkbox (default: unchecked) for storing protected variables. People have been happily using this feature for years. The only problem is, it's not standardized, so when you switch tool, it doesn't work. Why don't we just start standardizing stuff that is well-established? This doesn not preclude doing something fancier later.
Sure, I was just saying that we should think beyond this as well.
* To me not storing at events is an indication that you have something special causing "events" (such as discretized variables for a clock) where some more specialized solution would be better suited.For some users (e.g. @luizmac in OpenModelica/OpenModelica#5709) the requirement is to get periodically sampled outputs, to which you can, e.g., apply FFT analysis or discrete-time system identification techniques. In all such cases, the requirement is to have exactly one data point after every Interval of time, and nothing else.
To me the FFT (etc) should ideally pre-process the data to get that. That's at least what you can do in Dymola, so that you don't have to simulate it once for nice plotting and once for FFT-analysis.
This requirement is absolutely reasonable and I believe it would deserve an experiment annotation.
* In the linked issue the problem is how to capture transients and normal simulations in an efficient way. To me "non-equidistant grid" is not ideal for that, but kind of works. One possibility (implemented in Dymola) is to manually modify the grid rate during the simulation, but there might be more automatic ways.I don't know what other tools do for these cases.
OpenModelica at the moment is providing what solvers like DASSL or IDA already provide, i.e., either they internally advance the simulation with a variable step size solver and return periodically sampled solutions via dense output (what has been informally called "Equidistant time grid" so far, or they just return the solution at the steps taken by the solver, which is what you get if you switch off "Equidistant time grid". I agree with @HansOlsson that "No Equidistant time grid" kind of works when systems have fast transients every now and then, because you get more tight time points where you need them. But in some cases it gives serious aliasing problems, see the discussion in this ticket.
One option that has been put forth is to output the solution both at regular intervals and at variable time steps, though I understand this feature is not available natively in solvers such as IDA.
I was also considering this.
I only know Dymola and OpenModelica; both have a checkbox (default: unchecked) for storing protected variables.
To add to the list, System Modeler also has that (with the same default).
Language group meeting:
Additional ideas from System Modeler:
- Event hysteresis scaling; not in Dymola
- Non-solver tolerance (separate from simulation tolerance); not in Dymola
Storing protected seems like a good idea. Improved equidistant needs investigation.
Note: From a Dymola perspective it is deliberate choice to not expose the settings for event hysteresis and non-linear solver tolerance (even if they at some point in time existed), as they are no longer fixed settings.
I thus think we can get agreement on setting for storing protected, and non-equidistant if we reformulate it to indicate that one wants to store transients i.e., more frequently than the equidistant grid - but indicate it is a quality of implementation to also store the equidistant grid.
If people don't want the combination grid it seems obvious that you can enable this and set the interval to be equal to the simulation interval to effectively ignore the grid.
I thus think we can get agreement on setting for storing protected, and non-equidistant if we reformulate it to indicate that one wants to store transients i.e., more frequently than the equidistant grid - but indicate it is a quality of implementation to also store the equidistant grid.
If people don't want the combination grid it seems obvious that you can enable this and set the interval to be equal to the simulation interval to effectively ignore the grid.
I'm not sure I understand what the proposal is, ultimately. From my perspective, there are at least three relevant use cases:
- Regular time grid, no exceptions: this is needed, e.g., if I need data to do system identification or machine learning that assumes a sampled-data discrete-time model
- Regular time grid, plus extra points at events: this is needed to see exactly when events take place
- Solver time grid (a.k.a. no-equidistant time grid): time points are saved when the solver computes steps. That also include event times, since the solver processes them as well
As far as I understand, these are easily supported.
There is a fourth use case, which is
- Regular time grid + solver time grid: this would ensure that enough time points are saved when long time steps are taken, e.g. to avoid aliasing for oscillatory algebraic variables, but also ensure that in case of fast transients, they are correctly recorded and are not missed by a too large time step.
Unfortunately, it is not clear to me if such an option is readily implemented by integrators such as DASSL or IDA - I guess they'd need to be modified at the source code level to provide this feature.
I can provide four MWEs demonstrating the user requirements in all these cases.
I thus think we can get agreement on setting for storing protected, and non-equidistant if we reformulate it to indicate that one wants to store transients i.e., more frequently than the equidistant grid - but indicate it is a quality of implementation to also store the equidistant grid. If people don't want the combination grid it seems obvious that you can enable this and set the interval to be equal to the simulation interval to effectively ignore the grid.
I'm not sure I understand what the proposal is, ultimately. From my perspective, there are at least three relevant use cases:
- Regular time grid, no exceptions: this is needed, e.g., if I need data to do system identification or machine learning that assumes a sampled-data discrete-time model
- Regular time grid, plus extra points at events: this is needed to see exactly when events take place
- Solver time grid (a.k.a. no-equidistant time grid): time points are saved when the solver computes steps. That also include event times, since the solver processes them as well
As far as I understand, these are easily supported.
There is a fourth use case, which is
- Regular time grid + solver time grid: this would ensure that enough time points are saved when long time steps are taken, e.g. to avoid aliasing for oscillatory algebraic variables, but also ensure that in case of fast transients, they are correctly recorded and are not missed by a too large time step.
Unfortunately, it is not clear to me if such an option is readily implemented by integrators such as DASSL or IDA - I guess they'd need to be modified at the source code level to provide this feature.
I can provide four MWEs demonstrating the user requirements in all these cases.
To me the first use-case is something that should be handled in those routines - as part of processing the simulation result. Having a system identification package that can read result files, but not regularize the time-grid just seems odd. To me it would also be inconvenient to re-simulate the model with that change merely to remove event points.
I can understand that you in some cases don't want to store frequent events, but that is more an issue about not storing specific types of events than not storing all events.
As for the difference between the last two I agree that supporting regular time grid + solver time grid may take some effort. However, from the user perspective I find that more logical than having an interval-setting for solver time grid that doesn't influence the grid of the result in that case, but still may impact solver accuracy in odd ways (for DASSL). Given that you can in practice disable the regular time grid by setting it to simulation interval I don't see why having these as separate settings make sense.
- Regular time grid + solver time grid: this would ensure that enough time points are saved when long time steps are taken, e.g. to avoid aliasing for oscillatory algebraic variables, but also ensure that in case of fast transients, they are correctly recorded and are not missed by a too large time step.
Isn't this use case more naturally handled by defining an upper bound on the output grid step size? This is easily implemented similar to the handling of time events. Compared to overlaying the regular time grid, it has the advantage of never introducing a grid point very close a point where the solver has already been forced to stop due to some other event, resulting in a more even impact on step size computations and avoiding potential complications due to the solver rejecting too short integration intervals.
Edit: By the way, both DASSL and CVODE support specifying an upper bound on the solver step size, so the implementation is trivial with these solvers. In System Modeler, we call this __Wolfram_MaxStepSize in the experiment annotation.
- Regular time grid + solver time grid: this would ensure that enough time points are saved when long time steps are taken, e.g. to avoid aliasing for oscillatory algebraic variables, but also ensure that in case of fast transients, they are correctly recorded and are not missed by a too large time step.
Isn't this use case more naturally handled by defining an upper bound on the output grid step size? This is easily implemented similar to the handling of time events. Compared to overlaying the regular time grid, it has the advantage of never introducing a grid point very close a point where the solver has already been forced to stop due to some other event, resulting in a more even impact on step size computations and avoiding potential complications due to the solver rejecting too short integration intervals.
Yes, saying that the next output point is at the end of the step or after interval length - which-ever comes first; and then repeating after storing.
Edit: By the way, both DASSL and CVODE support specifying an upper bound on the solver step size, so the implementation is trivial with these solvers. In System Modeler, we call this
__Wolfram_MaxStepSizein theexperimentannotation.
That is different, as instead of taking a long step and the interpolate solutions during it - this just prevents the long step. Assuming the solver isn't broken it would roughly give similar results, but less efficiently.