cds-dbs icon indicating copy to clipboard operation
cds-dbs copied to clipboard

Expand compliance test suite

Open BobdenOs opened this issue 1 year ago • 0 comments

Goal

The goal of this PR is to extend the compliance test suite with more scenarios. While also cleaning up the existing compliance tests into a structured method of testing.

Reporter

When a test throws an error with the following pattern it will be considered to be part of the compliance test suite and will be put into the compliance reporter result.

  • The message starts with #
  • The message use the # character as a separator for category levels

This way it is possible to quickly generate a markdown file for a database service that precisely describes all the gaps that the database service has.

Example

Errors
# Type # Date # Does not support '1970-01-01'
# Expression # `>` # Does not consider 1 to be larger then 0
Report
  • {database}
    • Type
      • Date
        • Does not support '1970-01-01'
    • Expression
      • >
        • Does not consider 1 to be larger then 0

Specification

While extending the compliance test suite a lot of assumption on the specifications will be made. The goal is to have all the compliance tests reviewed and thereby a specification is gradually finalized. Including specification documentation as comments next to the test definition and when required links to external resources to provide full clarity. If we use an ISO or IEEE standard make sure to add a link to the detailed specification.

BobdenOs avatar May 30 '23 11:05 BobdenOs