quamina icon indicating copy to clipboard operation
quamina copied to clipboard

Draft a specification?

Open jsmorph opened this issue 1 year ago • 14 comments

I thought I'd create an issue for any on-going discussion:

As we've mentioned here and sort of in #57, it might be good to have some sort of specification for the pattern language. The current documentation is of course an excellent source, but adding something more formal is appealing (to me).

Ideally the specification is flexible enough to support something like feature flags ("shellstyle not supported" or whatever). Good test cases: Evolution of capabilities here (for example: pre-exists and post-exists specs) and similarly for Event Ruler.

This table from the EventBridge docs is of course helpful:

image

A potentially interesting twist is having the specification(s) not tied to specific event rendering languages (like JSON or protobuf), but that goal seems like a stretch with questionable net worth at this point.

Thoughts? Any appetite for something like this?

References

  1. https://github.com/timbray/quamina/blob/main/PATTERNS.md
  2. https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-event-patterns.html
  3. https://github.com/aws/event-ruler/blob/main/README.md
  4. https://en.wikipedia.org/wiki/Comparison_of_regular_expression_engines
  5. https://www.rfc-editor.org/rfc/rfc8259.html
  6. https://jsontypedef.com/
  7. https://json-schema.org/ (but I'm personally not a huge fan)

jsmorph avatar Mar 22 '23 11:03 jsmorph

Presumably if we're interested in a stable standards doc, the goal is interoperability with other implementations, of which at the moment there is one, namely aws-event-ruler. So I think that if we were going to do this, we should file an issue over there to alert them and invite them to collaborate. Sound reasonable?

While I would be happy to pitch in and help, I don't have the cycles to lead such an effort.

timbray avatar Mar 23 '23 05:03 timbray

This came up several times in our (AWS) discussions as well, paging @nikp and @baldawar

embano1 avatar Mar 23 '23 07:03 embano1

I'd be happy to contribute in whatever form. In the early days here, I had started a crude IETF RFC draft somewhere. I didn't get very far, but I'll see if I can dig it up.

jsmorph avatar Mar 23 '23 13:03 jsmorph

yeah lets collaborate on this. Is https://github.com/timbray/quamina/blob/main/PATTERNS.md a good place to converge on for now?

baldawar avatar Mar 23 '23 16:03 baldawar

If by "converge on" you mean "start from a copy of and preserve the RFC-like style" then I agree.

timbray avatar Mar 23 '23 16:03 timbray

I've never written one of these before but would be keen to help. CC @jonessha as well :)

nikp avatar Mar 23 '23 17:03 nikp

@timbray, what do you suggest for the drafting format these days? I'm not current at all, but drafting in Markdown might work well enough for us here. My weakly held preference is to start with whatever real markup, boilerplate, etc. from the outset (with content from PATTERNS.md) rather than retrofit later. What guidance do you have? @embano1 @baldawar @nikp and others, what say you?

I'm happy to start a branch (or whatever) with a crude first draft.

jsmorph avatar Mar 23 '23 18:03 jsmorph

Standards-wise, these days I live in the IETF. There, more or less all RFCs are created in either Markdown or XML. The tooling is pretty good either way. Have a look at the setup in https://github.com/ietf-wg-jsonpath/iregexp, if you clone that you get a Makefile and everything. If we have an explicit goal to take this to a standards organization we should pick one sooner rather than later because they all have strong opinions about what input drafts should look like.

My instinct would be to do it as an IETF internet-draft, because of that tooling, and anyone can post an Internet-draft for free, which gives at least a permanent URL.

Hey AWS guys, is there any relationship with CNCF these days? I sort of did that for a while but ran out of patience with them.

Oh, @jsmorph,d to answer your question: Yeah, markdown is fine.

timbray avatar Mar 23 '23 22:03 timbray

Yep, new repo feels appropriate to me, wonder what the AWS folks think.

On Thu, Mar 23, 2023 at 4:11 PM Jamie Stephens @.***> wrote:

@timbray https://github.com/timbray New repo for the internet-draft since we have two implementations?

— Reply to this email directly, view it on GitHub https://github.com/timbray/quamina/issues/210#issuecomment-1482029111, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAEJE7MJOOPLUGWJW7X3QLW5TKCFANCNFSM6AAAAAAWDWHSKQ . You are receiving this because you were mentioned.Message ID: @.***>

timbray avatar Mar 24 '23 03:03 timbray

New repo makes sense.

For full disclosure, so far I'm only speaking on my own behalf. I need to check if there's any due-process that needs to be followed before I work on behalf of AWS. From past experience, these things can take time so I wouldn't recommend waiting on it. I'd be keen to get started. Happy to create a repo w all permissions & scaffolding based on https://github.com/ietf-wg-jsonpath/iregexp over the weekend.

baldawar avatar Mar 24 '23 21:03 baldawar

on

Hey AWS guys, is there any relationship with CNCF these days? I sort of did that for a while but ran out of patience with them.

Sorry didn't see this before. I'm not sure at the moment. Let me ask in our internal channels.

baldawar avatar Mar 24 '23 21:03 baldawar

Hey AWS guys, is there any relationship with CNCF these days? I sort of did that for a while but ran out of patience with them.

I'm still a bit involved in the CloudEvents side (mostly for the Go and PowerShell SDKs). In general, we do have several streams with the CNCF. Which specific project is this question about?

embano1 avatar Mar 25 '23 06:03 embano1

I've created this shell repo temporarily https://github.com/baldawar/draft-ietf-patterns-protocol/ based on https://github.com/ietf-wg-jsonpath/iregexp/ & https://github.com/martinthomson/i-d-template (neat tool, btw!)

Everyone who I could think be interested should be invited to the repo (cc @longzhang-lz for context on the mysterious invite). I will move this to an organization as soon as someone can suggest a decent name. I could only come up w patterns / matchers / ruler patterns which are all felt too generic.

Right now I'm heads down on few work-related gremlins, but I can follow up within AWS on if there's any prescribed guidelines the company requires. I also have a bit of backlog of clean up work around event-ruler that I'm desperate to wrap up first, so expecting to take a lot more active role. Until then (targeting Apr End) happy to take care of any minor tasks, administrative duties, and reviews.

baldawar avatar Mar 25 '23 19:03 baldawar

I'll be starting to push out some PRs on https://github.com/epml-spec (Event Pattern Matching Language) where I've moved the repo.

baldawar avatar Apr 19 '23 05:04 baldawar