protobuf-grammar icon indicating copy to clipboard operation
protobuf-grammar copied to clipboard

Add context free proto file grammar

Open ObsidianMinor opened this issue 5 years ago • 5 comments

This repository is looking lacking, so I thought I'd get it started. I also plan to write specs for proto2 and proto3, but in the meantime I decided to write this context free grammar to document a grammar that works for every proto file regardless of whether the syntax is valid in context (like incorrect syntax versions or invalid values for certain options)

ObsidianMinor avatar Aug 24 '19 11:08 ObsidianMinor

Was there a reason this never got merged?

derekperkins avatar Feb 20 '20 02:02 derekperkins

+1, Was there a reason this never got merged?

teijeong avatar Jul 17 '20 02:07 teijeong

ObsidianMinor, we were going to turn down this repository when we discovered this outstanding PR. Sorry for the losing track of this!

Would you mind redirecting the location for the file to https://github.com/protocolbuffers/protobuf/blob/master/docs/grammar.ebnf? I'll work to get the doc reviewed by our engineering team in the meantime, in case any changes are needed before we can accept the PR.

Logofile avatar Feb 01 '22 18:02 Logofile

FWIW, here's another alternative that describes the language, also in EBNF. https://github.com/jhump/protocompile/blob/master/grammar/README.md

Unlike this document, it separates lexical analysis from the rest of the grammar productions in order to describe the nuance in tokenization relating to handling of whitespace and comments. It also has a slightly different way to interpret numeric literals, in an attempt to codify some of protoc's behavior. For example, in the face of "1to1000", the grammar in this PR suggests a tokenization of "1", "to", "1000", but protoc considers this a syntax error.

I have high confidence in the grammar as it is based on a yacc grammar that powers what I think might be the closest implementation to protoc itself.

jhump avatar Feb 01 '22 19:02 jhump

Joshua, if you'd like to submit a PR for the same target location (so we can get all of the CLAs and such covered), we'd love to take a look at adding it to our docs. I'm still coming up-to-speed on EBNF, so I'll need to get some SWE eyes on it once the PR comes in if you do submit one.

Logofile avatar Feb 01 '22 20:02 Logofile