kong icon indicating copy to clipboard operation
kong copied to clipboard

fix(core/worker_events): overwrite the `post` method of worker_events

Open liverpool8056 opened this issue 2 years ago • 5 comments

Summary

There is a limit for the size of payload allowed to be send in events lib. In some cases, Kong needs to send big payload. For example DAO:crud rely on events to invalidate cache. In these cases, the size of the payload may exceeds the limit due a large dao entity. In this PR, the post method of the worker_events is overwritten to take a retry action when we get the 'payload too big' error message, and the retry action is just to re-post the event with a truncated payload so that the subsequent logic that relies on the event could have a chance to be informed rather than unaware at all. Based on this purpose, the retry action is taken and only taken once when the error happens.

The limit for the size of payload is defined inside the overwritten post method as a conventional value.

Checklist

  • [x] The Pull Request has tests
  • [x] There's an entry in the CHANGELOG
  • [ ] There is a user-facing docs PR against https://github.com/Kong/docs.konghq.com - PUT DOCS PR HERE

Full changelog

  • [Implement ...]

Issue reference

FTI-4963 KAG-1523

liverpool8056 avatar May 15 '23 02:05 liverpool8056

This one is waiting for a co-related PR https://github.com/Kong/lua-resty-events/pull/28 to be merged and bumped

liverpool8056 avatar May 15 '23 02:05 liverpool8056

lua-resty-events has bumped to 0.1.5. See: https://github.com/Kong/kong/pull/10883

chronolaw avatar May 17 '23 08:05 chronolaw

Could we put the tests spec/02-integration/01-worker_events_spec.lua into a proper directory?

chronolaw avatar May 24 '23 08:05 chronolaw

In a real world, where do we have this issue happening?

The description says:

DAO:crud rely on events to invalidate cache. In these cases, the size of the payload may exceeds the limit due a large dao entity.

But where exactly, and why DAO crud event can be so big? Should we make it a bit more intelligent? E.g. truncate the fields or something? Or fix it at the source of event?

bungle avatar May 29 '23 07:05 bungle

In a real world, where do we have this issue happening?

For some plugin objects whose fields are not size-limited may be too large. As FTI-4963 suggested, the config field of request-validator plugin is too large for worker_event lib to send events. So large events are basically caused by size-unlimited fields.

Should we make it a bit more intelligent? E.g. truncate the fields or something?

I'm afraid it would be a bit flaky to try to truncate large fields of an entity, fields to be truncated might vary with the specific entities.

Or fix it at the source of event?

Fixing it at the source of event means we have to limit the size of fields that are unlimited initially, this would introduce a breaking change which might not such necessary at this moment.

liverpool8056 avatar May 29 '23 08:05 liverpool8056

This PR could be closed as we had another approach for this, referring to https://github.com/Kong/lua-resty-events/pull/37

liverpool8056 avatar Jul 12 '23 08:07 liverpool8056