pulsar
pulsar copied to clipboard
[wip][fix][fn] Add function name and id to producer name to avoid conflict
Fixes #21911
Main Issue: #xyz
PIP: #xyz
Motivation
there will be producer name conflicts error when function enable effectively_once and subscribed to a partitioned input topic
Modifications
Add functionId and instanceId to the suffix of the producer name to avoid conflicts
Verifying this change
- [x] Make sure that the change passes the CI checks.
- [ ]
- [x] This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
- [ ] Dependencies (add or upgrade a dependency)
- [ ] The public API
- [ ] The schema
- [ ] The default values of configurations
- [ ] The threading model
- [ ] The binary protocol
- [ ] The REST endpoints
- [ ] The admin CLI options
- [ ] The metrics
- [ ] Anything that affects deployment
Documentation
- [ ]
doc
- [ ]
doc-required
- [x]
doc-not-needed
- [ ]
doc-complete
Matching PR in forked repository
PR in forked repository: https://github.com/jiangpengcheng/pulsar/pull/21
Thanks for your fix. I have a question. After adding the instance name to producerName, can the effectively_once be guaranteed? For example, there was no ack for the message before the failure, but the message was resent after switching to another instance. I think deduplication is invalid at this time because of different producerNames. @jiangpengcheng
@graysonzeng hm, you are right, let's consider an another way to fix the issue
Codecov Report
Attention: 1 lines
in your changes are missing coverage. Please review.
Comparison is base (
24c927e
) 73.60% compared to head (247c4af
) 72.39%. Report is 12 commits behind head on master.
Additional details and impacted files
@@ Coverage Diff @@
## master #21912 +/- ##
============================================
- Coverage 73.60% 72.39% -1.21%
+ Complexity 32359 32030 -329
============================================
Files 1859 1861 +2
Lines 138373 143467 +5094
Branches 15160 16278 +1118
============================================
+ Hits 101853 103870 +2017
- Misses 28640 31483 +2843
- Partials 7880 8114 +234
Flag | Coverage Δ | |
---|---|---|
inttests | 24.28% <0.00%> (+0.11%) |
:arrow_up: |
systests | 23.67% <0.00%> (-0.06%) |
:arrow_down: |
unittests | 71.53% <75.00%> (-1.35%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
Files | Coverage Δ | |
---|---|---|
...a/org/apache/pulsar/functions/sink/PulsarSink.java | 73.75% <75.00%> (-0.10%) |
:arrow_down: |