Subscribing to events [EPIC]

[k15r]

As a user of kyma eventing i want to be able to subscribe to events by specifying only the information i really want to filter events by.

For me this means:

• an event sent by a connected application can be specified in a filter by specifying:
  • the application it was initially sent from
  • the event-type contained in my original event
• an event sent from within the cluster:
  • event-type
  • internal source of the event (name of the lambda eg, or some user configurable identifier) (optional)
• an event not sent using the eventpublisher.
  • a full eventtype that adheres to the backend the subscription is working on. This will allow me to subscribe to events that are not sent using kyma, but are directly sent to the eg the event mesh.
  • the external source identifier

Tasks

• [x] Proposal for refactoring eventing subscription CRD filters #14552
  • [ ] List of Tasks
• [x] Proposal for the revision of subscription status in the new Subscription CRD
• [x] Migration/Upgrade part for releasing the new CRD
• [ ] Generate and maintain certificates for the eventing webhook server
• [x] #15518
• [ ] #15684
• [ ] #15521
• [x] #15362
• [ ] #15363
• [x] #15364
• [ ] #14750
• [ ] #15685
• [ ] #15618
• [ ] #15622
• [ ] #15586
• [ ] #15624
• [ ] #15638
• [ ] #15688

[friedrichwilken]

Let's say you want to create a subscription like this, where would the info mentioned before go?

apiVersion: eventing.kyma-project.io/v1alpha1
kind: Subscription
metadata:
  name: test-noapp
  namespace: goldfish-testing
spec:
  filter:
    filters:
    - eventSource:
        property: source
        type: exact
        value: ""
      eventType:
        property: type
        type: exact
        value: sap.kyma.custom.noapp.order.created.v1
  protocol: ""
  protocolsettings: {}
  sink: http://test.goldfish-testing.svc.cluster.local
EOF

[k15r]

One important thing to know about the current state of the subscription is, that it was modelled based on the EventMesh subscription definition. If do not expose that one to the customer, but just use it internally for subscriptions on EventMesh we can change our subscription crd:

apiVersion: eventing.kyma-project.io/v1alpha1
kind: Subscription
metadata:
  name: test-noapp
  namespace: goldfish-testing
spec:
  filter:
    filters:
    - sourceType: connectedApp
      source: <application>
      eventType: <eventType>
    - sourceType: internal
      source: <internalsource>|optional
      eventType: <eventType>
    - sourceType: external
      eventType: <full event type>
      source: <externalSource>
  sink: http://test.goldfish-testing.svc.cluster.local

Obviously not all backends will support the external Source (eg Nats Jetstream)

[nachtmaar]

@k15r I did just read the story and came across the original story. I found these todos in the ticket which were quite helpful for me:

• [ ] document subscribing to events from applications directly connected to ems (eg. sap.c4c.foo.created.v1)
• [ ] rethink automatic type generation approach
• [ ] create a proposal for a more flexible solution

---

The todos answer a few questions which I had:

• ~What is the outcome of the ticket ?~ -> a proposal
• I don't think I ever saw the example where events are directly sent to Event Mesh and consumed in Kyma. I think this would help me to understand the issue even better. Maybe this is the first thing we need to do then in the proposal

[nachtmaar]

@k15r

Obviously not all backends will support the external Source (eg Nats Jetstream)

What do you mean by this ? That I cannot send events to Event Mesh and consume them with the Jetstream backend ? 🤔

[nachtmaar]

Result from last grooming discussion

We discussed that we want to write a proposal first before jumping into any implementation since this needs some 🧠 power. Since it is hard to describe what the outcome of the proposal is we had the idea that the proposal shall provide us answers to questions and we just need to come up with the questions:

Here are some questions which came to my mind:

• What are the current ways to subscribe to events / which rules does eventing have to follow in these cases:
  • Cases:
    • external event (via gateway, https://gateway./myapp/events)
    • internal event (via event-publisher-proxy)
    • events sent directly to the underlying eventing system (e.g. Event Mesh)
    • NATS vs. Event Mesh
  • Reasoning: This is a good introduction to the proposal and gives the reader an overview over the current state before proposing a better solution.
• How would eventing look like without the limitations/validations which we only do for Event Mesh ?
  • or formulated differently: Which information are only required by the user to subscribe to events and which information are required only by the backend and could be inferred/created by us ?
  • Reasoning: This can help us get an idea of what we can make easier for the user.
• What is required in order to jump into the implementation of the task ?

Scope of the proposal (for me)

• Merging the proposal on https://github.com/kyma-project/community means for me that:
  • Follow-up tickets for the implementation of this epic are created.
  • In case the proposal comes up with multiple options:
    • Then there should be team agreement on what is the preferred option.
    • The pros and cons of the different options should be described to help the reader understand the options.

[nachtmaar]

More questions:

• [ ] in case we want to remove event-mesh related data from subscription CRD to source code. how can this be achieved ?