zig-yaml icon indicating copy to clipboard operation
zig-yaml copied to clipboard

YAML 1.2 compatibility tests

Open kubkon opened this issue 2 years ago • 4 comments

It would be very cool if we subjected the parser to a YAML 1.2 compatibility tests (if such even exist).

kubkon avatar Nov 22 '22 16:11 kubkon

if such even exist).

Yep this exists: https://matrix.yaml.info/ https://github.com/yaml/yaml-test-suite

I had thought to suggest this as well! (and add zig-yaml to the compatibility matrix) If I can ever find some time I would help setting these up.

adamserafini avatar Dec 13 '22 13:12 adamserafini

Hello @kubkon and @adamserafini , I needed a YAML parser so I checked this out and found these issues and went down a rabbit hole.

Fixed up the parser so it runs with 0.11.0-dev.1026+4172c2916

I have now added some work to the build.zig which checks for the existence of the 'data' folder that is generated via the yaml-test-suite inside the 'test' folder. If it finds this, it then auto generates a file full of tests that look like this

`test "indent/DK95/08" {
    if(loadFromFile("test/data/tags/indent/DK95/08/in.yaml")) |yaml_const| {
        var yaml = yaml_const;
        yaml.deinit();
        try testing.expect(true);
    } else |_| {
        try testing.expect(false);
    }
}

test "indent/DK95/01" {
    if(loadFromFile("test/data/tags/indent/DK95/01/in.yaml")) |yaml_const| {
        var yaml = yaml_const;
        yaml.deinit();
        try testing.expect(false);
    } else |_| {
        try testing.expect(true);
    }
}`

Essentially just checking if we can load the YAML file without it crashing. In the case of indent/DK95/08 it expects true if we parse correctly and false on an error. In the case of indent/DK95/01 It expects an error (denoted by presence of 'error' file in folder).

This is all I have achieved so far, currently it sits at 349 passed; 0 skipped; 801 failed.

Feel free to check my WIP at

https://github.com/Bergasms/zig-yaml

Bergasms avatar Jan 29 '23 14:01 Bergasms

Very cool work @Bergasms! Wanna create a PR and upstream your work? I will happy to code review your changes. One thing that I would suggest based on the snippet you posted is for the generator to emit something like:

test "indent/DK95/08" {
    var yaml = loadFromFile("test/data/tags/indent/DK95/08/in.yaml") catch return error.Failed; // Or even hone in on the error if desired
    defer yaml.deinit();
}

test "indent/DK95/01" {
    var yaml = loadFromFile("test/data/tags/indent/DK95/01/in.yaml") catch return; // Error is success so just return
    defer yaml.deinit();
    return error.UnexpectedSuccess;
}

kubkon avatar Jan 29 '23 14:01 kubkon

@kubkon Yes I will, I've been self teaching zig and failing forward, so to speak, so it will be good to have some guidance. I'll do a bit more to get it into a less 'hacked together' state and then put up a PR, when time permits.

Bergasms avatar Jan 29 '23 22:01 Bergasms