eventmesh
eventmesh copied to clipboard
[Enhancement] Define and standardize some common configurations for all Sources
Search before asking
- [X] I had searched in the issues and found no similar issues.
Enhancement Request
In the various implementations of Source
, there are often some common configurations scattered across different places, such as the capacity of the storage queue, the number of elements to poll each time, etc. Additionally, in some source implementations, the poll
method can block for up to several seconds in the worst-case scenario, reaching maxBatchSize * pollTimeout
. Therefore, it is necessary to define common parameters that support configuration and also limit the maximum waiting time for poll
.
Describe the solution you'd like
- Added a
PollConfig
configuration class that includes parameters forcapacity
,maxBatchSize
, andmaxWaitTime
, and incorporated it as a variable inSourceConfig
to support user configuration with default values. - Replaced the fixed constants for
capacity
,maxBatchSize
, andmaxWaitTime
in the original source implementations with the corresponding variables fromPollConfig
. - Optimized some
poll
methods to ensure the maximum waiting time does not exceedmaxWaitTime
, preventing blocking for several seconds.
Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct *