Validity of clocked signals in reference results

[maltelenz]

Reading a reference result with variables that are clocked, like this, what part of that file can or should be trusted/used for correctness checking?

A: All points B: Only the right hand side of the first clock tick (duplicated time point) and all the remaining ticks (both sides) C: Both sides of all clock ticks (duplicated time points) D: Some other answer (please explain)

Related: #3516

[henrikt-ma]

I suppose one needs to consider that a duplicated time point isn't necessarily a clock tick; it could be a normal time or state event too. If a clock tick results in a duplicated time point, I suppose it is also totally normal that a duplicated time point belongs to an unrelated clock.

[maltelenz]

I suppose one needs to consider that a duplicated time point isn't necessarily a clock tick; it could be a normal time or state event too. If a clock tick results in a duplicated time point, I suppose it is also totally normal that a duplicated time point belongs to an unrelated clock.

That is true, and a fact that seems unavoidable with the simple CSV format our baselines are in.

For the sake of discussion, lets assume we know where ticks occur for each variables associated clock.

[henrikt-ma]

I find it very difficult to address this as a question about result file interpretation, as long as there is no clear answer to the language specification question whether a clocked variable can have a value before the first tick of its associated clock, see #3516. If so, we would need to know for which variables this is true.

If a clocked variable doesn't have a value before the first tick, I see two alternatives for how to interpret the result file values before the first clock tick:

• Undefined; must be ignored.
• The variable's start attribute; useful for comparison, but possibly confusing since we are storing something which strictly speaking isn't a value of the variable's trajectory. This also has an adverse impact on alias handling, see below.

By defining the part before the first clock tick as the variable's start attribute, one destroys the possibility to treat aliasing of clocked variables efficiently, since – unless their start attributes are equal – they can't share trajectory in a result file format with support for aliasing. To me, this seems like a rather high price to pay just in order to have a representation in result files of something that strictly speaking isn't considered a value belonging to the variable's trajectory.

[christoff-buerger]

If a clocked variable doesn't have a value before the first tick, I see two alternatives for how to interpret the result file values before the first clock tick:

Undefined; must be ignored.

That's the way to go in my opinion. W.r.t. the current Modelica specification, clocked variables have no value before the first clock tick, except if subject to previous, in which case also previous(v) has a value.

In Dymola for example, when plotting clocked variables, we do not plot anything until the first tick.

I don't know how we can incorporate this undefined value behaviour in our current test infrastructure though.

[henrikt-ma]

@maltelenz is currently on vacation, but when he is back I think he might have some ideas to share about how to handle this in "our current test infrastructure" which I suppose refers to the reference result files provided for the MSL examples.

Internally at Wolfram we already have infrastructure for proper validation of clocked variable results. The exact hows of the proper comparison is the topic of this issue (although the question of what is defined and what isn't should ultimately be resolved in the scope of the specification, not the standard library), but as far as I can tell we are currently using it successfully to only compare values of clocked variables at their clock ticks.

[maltelenz]

I believe the issues brought up in the comments above, as well as my experience in implementing proper checking of clocks/clocked variables for System Modeler, makes it clear the CSV files we have as the "current test infrastructure" are not rich enough to represent all you need to do proper verification of clocks and clocked signals.

For System Modeler, we use json files for reference/trial data. The format for the synchronous data we currently use solves both issues raised in this issue:

• Optional "before first tick" value
• Clear when a clock has ticked, cannot be confused with unrelated events

Our format basically has entries similar to these for clocked variables and their clocks, respectively:

      "someClockedVar": {
        "values": [ 10.1, 10.2, 10.3, 10.4, 10.5, 10.6, 10.7, 10.8, 10.9 ],
        "clock": "someClock",
        "start": 1.0 // This is an optional item. If it is not present, it means the clocked variable has no value before the first clock tick
      },
...
      "someClock": {
        "ticks": [ 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0 ]
      }

I believe we need something like this, which is more expressive, if we are serious about clocked references and verification.