Kamelet CloudEvents set `source` attribute to the string "source"

[lance]

When using a Twitter search source kamelet (and I assume it's applicable to all kamelets), I have found that the CloudEvent headers, while adhering to the CE spec, don't necessarily adhere to the spirit. Specifically, with the source attribute, the value provided is source. The spec specifies the source field is a URI reference. We could argue that source is a relative URI, but it's definitely not the actual source of the event.

Identifies the context in which an event happened. Often this will include information such as the type of the event source, the organization publishing the event or the process that produced the event. The exact syntax and semantics behind the data encoded in the URI is defined by the event producer.

It would be much nicer if the source attribute could be used to identify the event producer uniquely.

The type attribute could also be improved. That value is currently org.apache.camel.event which is pretty good. It adheres to the CE spec recommendation.

SHOULD be prefixed with a reverse-DNS name. The prefixed domain dictates the organization which defines the semantics of this event type.

But really all we are getting with this value is the prefix. It would be great if, for example, the type could be something that would allow more finely grained filtering. E.g. org.apache.camel.event.twitter.search for a Twitter search kamelet, and org.apache.camel.event.twitter.timeline for a Twitter timeline kamelet.

🙏

[lburgazzoli]

I had a look at the implementation and I need some more information about how to move forward.

In order to set the context, a KameletBinding is an high level abstraction that is rendered to a number of Camel routes so assuming you have a from Twitter to Knative binding as in this case, like:

apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
  name: twitter-to-channel
spec:
  source: (1)
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1alpha1
      name: twitter-search-source
    properties:
      keywords: "foo,bar"
      accessToken: "the-token-here"
  sink: (2)
    ref:
      kind: InMemoryChannel
      apiVersion: messaging.knative.dev/v1
      name: tweets

Then it gets roughly translated in something like:

from("twitter-search:foo,bar?accessToken={{accessToken}}") //1
    .routeId("source")
    .to("direct:sink");

from("direct:sink") //2
    .routeId("sink")
    .to("knative:channel/tweets");

What the camel-knative component does when setting up the cloud event attributes, is to retrieve the route id that has generated the message and bound it to the CloudEvent source filed and according to the example above, then the value will be source.

It could be possible to retrieve the original from uri to set the source field, however this pose some questions as the uri may contain information that are not needed but most important, placeholders and/or sensitive info which I think should not be exposed.

Also what would be the best here for composing the source ? For this case, having something like twitter-search:foo,bar would probably be good enough to satisfy the use case but in some cases, additional information about the source are part of the camel URI's query parameters so we would need a more complex way to compute the source according to the spec.

It looks to me that this is another use case for https://github.com/apache/camel-k/issues/3501

@lance is it ok to have non unique sources ? i.e. if you create two from Twitter to Knative bindings for the same set of keywords but for different accounts.

[lance]

In order to set the context, a KameletBinding is an high level abstraction that is rendered to a number of Camel routes so assuming you have a from Twitter to Knative binding as in this case, like...

Thanks for this explanation.

It could be possible to retrieve the original from uri to set the source field, however this pose some questions as the uri may contain information that are not needed but most important, placeholders and/or sensitive info which I think should not be exposed.

It is too bad that the URI may contain sensitive info because IMO this is the proper value for a source attribute.

@lance is it ok to have non unique sources ? i.e. if you create two from Twitter to Knative bindings for the same set of keywords but for different accounts.

Per the spec, I believe the combination of the the source and id attributes should be unique, but not the source alone. The description in the spec seems to be written for this use case. :)

A source MAY include more than one producer. In that case the producers MUST collaborate to ensure that source + id is unique for each distinct event.

[lburgazzoli]

@lance it will take a while but we are working to improve support for cloud events, this is the start https://github.com/apache/camel-kamelets/pull/1162

[github-actions[bot]]

This issue has been automatically marked as stale due to 90 days of inactivity. It will be closed if no further activity occurs within 15 days. If you think that’s incorrect or the issue should never stale, please simply write any comment. Thanks for your contributions!

[lance]

Just following up here so that the issue doesn't get closed, and to provide what is my current workaround for this issue. When creating a Knative Trigger, I inject CloudEvent source and type headers using the following steps in the KameletBinding.

apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
  name: telegram-source-binding
spec:
  source:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1alpha1
      name: telegram-source
    properties:
      authorizationToken: API_TOKEN_HERE
  steps:
  - ref:
      apiVersion: camel.apache.org/v1alpha1
      kind: Kamelet
      name: insert-header-action
    properties:
      name:  CamelCloudEventSource
      value: org.apache.camel.event.telegram
  - ref:
      apiVersion: camel.apache.org/v1alpha1
      kind: Kamelet
      name: insert-header-action
    properties:
      name:  CamelCloudEventType
      value: telegram.message
  sink:
    ref:
      kind: Broker
      apiVersion: eventing.knative.dev/v1
      name: default

This works fine, but it would be nice to have reasonable defaults so that the yaml can be as simple as possible.

[squakez]

@christophd I think this would be now covered by https://github.com/apache/camel-kamelets/blob/4.2.x/kamelets/data-type-action.kamelet.yaml. Can you confirm?