protoreflect icon indicating copy to clipboard operation
protoreflect copied to clipboard

protoc compatibility tests

Open WillAbides opened this issue 4 years ago • 1 comments

This adds tests for compatibility between protoc and protoparser.

It runs protoc with --descriptor_set_out and compares that output with the output of parser.ParseFiles() ignoring differences in SourceCodeInfo and RawFields. The test fails if there is a difference in the outputs or if one errors but the other doesn't.

The new tests will fail until #439, #440, #442 and #444 are fixed. If you want to not test those until they are fixed you can add knownIssueMatchers: knownIssueMatchers to the options here and then remove the matchers from here as issues are fixed. Either way the matchers should be removed as issues are fixed.

The compatibility tests requires that protoc is available. You can point it to a specific protoc with the PROTOC_BIN environment variable. If that isn't set it will search PATH for protoc. If it still isn't found it skips the test.

This also adds a fuzz test that tests for compatibility with protoc. You can run the fuzz test locally with make fuzz. Or if you don't want to use all your cores try GO_FUZZ_ARGS='--procs 2' make fuzz.

WillAbides avatar Nov 01 '21 21:11 WillAbides

If this is merged https://github.com/google/oss-fuzz/tree/master/projects/protoreflect should be updated with FuzzProtocCompatibility

WillAbides avatar Nov 01 '21 21:11 WillAbides

@WillAbides, any future changes to the parser should be done in https://github.com/bufbuild/protocompile. There are a large number of test cases in that repo that are compared against protoc automatically, by invoking protoc and requiring protoc to succeed if protocompile succeeds and vice versa. There are separate tests that check other low-level output details of the descriptors, including comparing against protoc's SourceCodeInfo output and even comparing the serialized bytes (a *linker.Result in the protocompile module has a way to serialize the file descriptor to "canonical bytes", which emulates the behavior of protoc when serializing options).

I am hoping to roll out a v2 soon of this repo, and it will not have a protoparse package: it has been superceded by protocompile. So I'm closing this PR. Sorry it was left open for so long with no discussion.

jhump avatar Dec 08 '23 00:12 jhump