Apply Limit to the payload when sending split completed events

[b-slim]

Description

The current pull request integrates the capability to enforce a JSON payload size limit when dispatching split-completed events, aligning its truncation behavior with that observed in query-completed events.

Additional context and related issues

In certain instances, Driver statistics may reach significant volumes, necessitating their exclusion similar to the procedure employed for query-completed events Changing the return type of payload to optional, To keep it consistent with the QueryCompleted event leads to a change in the SPI. Another option is to keep it as is and return a null String.

Release notes

() This is not user-visible or docs only and no release notes are required. ( ) Release notes are required, please propose a release note for me. (X ) Release notes are required, with the following suggested text:

Discard output stage JSON from split completion event when it is very long.
  This limit can be configured with `event.max-split-output-stage-size`

[raunaqmorarka]

Can you share an example of a large event ? Is there something specific inside DriverStats that gets too big in your observations ?

[phd3]

@raunaqmorarka OperatorsStats#info can be large depending on the operator - for example in a big cluster, the list of pagebufferclientstatus can get large for exchange operator.

[b-slim]

Can you share an example of a large event? Is there something specific inside DriverStats that gets too big in your observations?

Good question, the size is relative to this amount of work and size of clusters the bulk goes to this kind of updates.

{"uri":"http://ip:8080/v1/task/20230515_153301_35230_pnpyx.16.230/results/11","state":"closed","lastUpdate":"2023-05-15T15:34:18.599Z","rowsReceived":2,"pagesReceived":1,"requestsScheduled":6,"requestsCompleted":6,"requestsFailed":0,"httpRequestState":"not scheduled"},

In our infra is highly recommended to have a clear limit on the amount of metrics we send thus the need to have such limits.

[phd3]

@b-slim thanks, only a couple more comments. can you please squash commits as well?

[github-actions[bot]]

This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua

[mosabua]

@b-slim and @phd3 could you resolve conflicts and complete work on this PR. Seems like it is very close.

[github-actions[bot]]

This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua

[mosabua]

@b-slim can you rebase .. this is otherwise good to go.

[github-actions[bot]]

This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua

[github-actions[bot]]

Closing this pull request, as it has been stale for six weeks. Feel free to re-open at any time.