trino icon indicating copy to clipboard operation
trino copied to clipboard

implemented sending split completed events over to Kafka

Open domnikl opened this issue 6 months ago • 6 comments

Description

Issue: https://github.com/trinodb/trino/issues/23065

Implements sending SplitCompletedEvents to Kafka in the event listener plugin. By default those will not be sent to keep the current behavior of only sending query created and query completed.

Additional context and related issues

Example plugin configuration (extended with the new values for split completed):

event-listener.name=kafka-event-listener
kafka-event-listener.publish-created-event=false
kafka-event-listener.publish-completed-event=true
kafka-event-listener.publish-split-completed-event=true
kafka-event-listener.completed-event.topic=query_completed
kafka-event-listener.split-completed-event.topic=split_completed
kafka-event-listener.client-id=dominik.liebler.test
kafka-event-listener.broker-endpoints=kafka-host-1:9093,kafka-host-2:9093
kafka-event-listener.excluded-fields=ioMetadata,payload,groups,cpuTimeDistribution
kafka.security-protocol=plaintext

Release notes

( ) This is not user-visible or is docs only, and no release notes are required. ( ) Release notes are required. Please propose a release note for me. (x) Release notes are required, with the following suggested text:

# General
* Extend Kafka event listener plugin to send split completed events (https://github.com/trinodb/trino/issues/23065)

domnikl avatar Aug 19 '24 08:08 domnikl