Test suite

[shoogle]

We should create a suite of test cases that contains examples of valid and invalid StrictYAML, as well as the equivalent JSON or assciated error codes and messages. The test suite should go in a separate repository that parser implementations can include as a submodule.

This will help with:

• #49 creating a specification
• #94 cross platform grammar

and it would also aid and encourage implementation in other languages.

---

Example tests

Invalid StrictYAML

Files

E101.txt

E101: Newline characters in keys must be escaped

E101.yaml

multiline
key: value

Proceedure

To pass this test, an implementation must attempt to parse the StrictYAML file (which is invalid) and raise an exception with a message that includes the error code provided.

Valid StrictYAML

Files

norway.txt

Ensure "NO" is parsed as a string rather than as the boolean FALSE used in regular YAML.

norway.yaml

countries:
- GB
- FR
- NO

norway.json

{
    "countries": [
        "GB",
        "FR",
        "NO"
    ]
}

Proceedure

To pass this test, an implementation must:

1. Test parsing by reading the StrictYAML file and spitting out JSON that matches the provided file.
2. Test dumping by reading the JSON file and spitting out StrictYAML that matches the provided file.
3. Test roundtripping by reading the StrictYAML file and spitting out an identical file.

Note that in some cases the dumping test would not work:

• If the example StrictYAML file uses a unusual (but valid) indentation.
• If it has quotation marks or escape characters that are not strictly necessary.

In these cases we could provide a normalized version of the StrictYAML file that uses default quoting, escaping and indentation for use with the dumping test.

[crdoconnor]

I'm definitely up for doing this although I'd like to get a parser running first that does the basics.

On Wed, 22 Apr 2020, 05:24 Peter Jonas, [email protected] wrote:

We should create a suite of test cases that contains examples of valid and invalid StrictYAML, as well as the equivalent JSON or assciated error codes and messages. The test suite should go in a separate repository that parser implementations can include as a submodule.

This will help with:

• #49 https://github.com/crdoconnor/strictyaml/issues/49 creating a specification
• #94 https://github.com/crdoconnor/strictyaml/issues/94 cross platform grammar

and it would also aid and encourage implementation in other languages.

Example tests Invalid StrictYAML Files E101.txt

E101: Newline characters in keys must be escaped

E101.yaml

multilinekey: value

Proceedure

To pass this test, an implementation must attempt to parse the StrictYAML file (which is invalid) and raise an exception with a message that includes the error code provided. Valid StrictYAML Files norway.txt

Ensure "NO" is parsed as a string rather than as the boolean FALSE used in regular YAML.

norway.yaml

countries:

• GB
• FR
• NO

norway.json

{ "countries": [ "GB", "FR", "NO" ] }

Proceedure

To pass this test, an implementation must:

1. Test parsing by reading the StrictYAML file and spitting out JSON that matches the provided file.
2. Test dumping by reading the JSON file and spitting out StrictYAML that matches the provided file.
3. Test roundtripping by reading the StrictYAML file and spitting out an identical file.

Note that in some cases the dumping test would not work:

• If the example StrictYAML file uses a unusual (but valid) indentation.
• If it has quotation marks or escape characters that are not strictly necessary.

In these cases we could provide a normalized version of the StrictYAML file that uses default quoting, escaping and indentation for use with the dumping test.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/crdoconnor/strictyaml/issues/96, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOJKNJXHZEXLGAM7JUU36DRNZWQVANCNFSM4MN2J3MA .

[crdoconnor]

FYI i'm working on this currently. I haven't forgotten about this, but it might take a while longer.

[shoogle]

Probably best to put it in a separate repository as I said before. If you create this repo with the above example then other people can start to submit PRs with more tests.

[crdoconnor]

I'm doing that. It will be something that any parser from any language should be able to use (in theory).

On Tue, 11 Aug 2020, 19:15 Peter Jonas, [email protected] wrote:

Probably best to put it in a separate repository as I said before. If you create this repo with the above example then other people can start to submit PRs with more tests.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/crdoconnor/strictyaml/issues/96#issuecomment-672154471, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOJKNNB2WDRG2IWA4WDNKDSAGDFPANCNFSM4MN2J3MA .