guac icon indicating copy to clipboard operation
guac copied to clipboard

fix queue to deliver message directly

Open pxp928 opened this issue 1 year ago • 0 comments

Description of the PR

Previously all the messages were being grabbed from the queue and placed into a channel for processing. This resulted in one ingestor grabbing majority of the messages and slowing trying to drain the channels. With the direct receive of messages from the pubsub, the goCloud library can throttle the batch size as needed and each ingestor can ingest simultaneously. No functionality changes, just removed the necessary use of channels in the emitter.

PR Checklist

  • [x] All commits have a Developer Certificate of Origin (DCO) -- they are generated using -s flag to git commit.
  • [x] All new changes are covered by tests
  • [ ] If GraphQL schema is changed, make generate has been run
  • [ ] If OpenAPI spec is changed, make generate has been run
  • [ ] If collectsub protobuf has been changed, make proto has been run
  • [x] All CI checks are passing (tests and formatting)
  • [x] All dependent PRs have already been merged

pxp928 avatar Apr 14 '24 15:04 pxp928