RecordFlux icon indicating copy to clipboard operation
RecordFlux copied to clipboard

Coverage of specifications cannot reach 100%

Open rssen opened this issue 3 years ago • 1 comments

With the current implementation of the validator, it is not possible to reach 100% coverage for some specifications. There are 3 cases that lead to this problem:

  1. A specification imports another specification but does not define a refinement to messages of the imported specification. For example, the arp.rflx specification imports ipv4.rflx but only uses field types of ipv4.rflx, it does not define a refinement ARP -> IPv4 (of cause not). The output of the validation is shown below.
  2. A specification imports another specification and defines a refinement to only one message of the imported specification (if multiple messages are defined in the imported specification).
  3. A specification contains completely separate messages. Cf. #833
  4. (and combinations of the 3 cases)

Discussed suggestions so far:

  1. Compute the coverage per-message not per-specification file. As a downside, this could lead to forgotten messages.
  2. Only compute coverage of the PDU specification, do not include refinements. (i.e. when validating ARP::Packet_IPv4 only get coverage of arp.rflx). This would require some logic to include refinement messages that are an integral part of the PDU, as e.g. IPv4Options to IPv4. This would also make getting test data easier, as one would not need to search for specific combinations of PDU/SDU messages.
--------------------------------------------------------------------------------
                     RecordFlux Validation Coverage Report
Directory: /home/rs/Dev/RecordFlux-specifications
--------------------------------------------------------------------------------
File                                          Links       Used        Coverage
ipv4.rflx                                        27          0           0.00%
ethernet.rflx                                    12          0           0.00%
arp.rflx                                         10         10         100.00%
--------------------------------------------------------------------------------
TOTAL                                            49         10          20.41%
--------------------------------------------------------------------------------

rssen avatar Feb 15 '22 11:02 rssen

Related to #814

senier avatar Aug 25 '22 09:08 senier