Bumps the franz-go group with 2 updates: github.com/twmb/franz-go and github.com/twmb/franz-go/pkg/kadm.

Updates github.com/twmb/franz-go from 1.14.3 to 1.16.0

Changelog

Sourced from github.com/twmb/franz-go's changelog.

v1.16.0

This release contains a few minor APIs and internal improvements and fixes two minor bugs.

One new API that is introduced also fixes a bug. API-wise, the SaramaHasher was actually not a 1:1 compatible hasher. The logic was identical, but there was a rounding error because Sarama uses int32 module arithmetic, whereas kgo used int (which is likely int64) which caused a different hash result. A new SaramaCompatHasher has been introduced and the old SaramaHasher has been deprecated.

The other bugfix is that OptValue on the kgo.Logger option panicked if you were not using a logger. That has been fixed.

The only other APIs that are introduced are in the kversions package; they are minor, see the commit list below.

If you issue a sharded request and any of the responses has a retryable error in the response, this is no-longer returned as a top-level shard error. The shard error is now nil, and you can properly inspect the response fully.

Lastly (besides other internal minor improvements not worth mentioning), metadata fetches can now inject fake fetches if the metadata response has topic or partition load errors. This is unconditionally true for non-retryable errors. If you use KeepRetryableFetchErrors, you can now also see when metadata fetching is showing unknown topic errors or other retryable errors.

a2340eb improvement pkg/kgo: inject fake fetches on metadata load errors

d07efd9 feature kversion: add VersionStrings, FromString, V3_6_0

8d30de0 bugfix pkg/kgo: fix OptValue with no logger set

012cd7c improvement kgo: do not return response ErrorCode's as shard errors

1dc3d40 bugfix: actually have correct sarama compatible hasher (thanks @C-Pro)

v1.15.4

This patch release fixes a difficult to encounter, but fatal-for-group-consuming bug.

The sequence of events to trigger this bug:

OffsetCommit is issued before Heartbeat

The coordinator for the group needs to be loaded (so, likely, a previous NOT_COORDINATOR error was received)

OffsetCommit triggers the load

a second OffsetCommit happens while the first is still running, canceling the first OffsetCommit's context

In this sequence of events, FindCoordinator will fail with context.Canceled and, importantly, also return that error to Heartbeat. In the guts of the client, a context.Canceled error should only happen when a group is being

... (truncated)

Commits

c1a3c3d Merge pull request #664 from twmb/changelog-v1.16
b02fa4d CHANGELOG: note incoming v1.16
7584d23 Merge pull request #665 from twmb/patches-setoffsets-revert
405d87d Revert "pkg/kgo: remove SetOffsets restriction of "only previously polled par...
b45f393 kgo: show SetOffsets behavior problem
a8e33ff Merge pull request #662 from twmb/patches
0b3766d Merge pull request #622 from C-Pro/sarama-compat-partitioner
3e6d9f9 Merge pull request #663 from twmb/go1.19
636b45c kgo testing: add UnknownTopicRetries(-1)
c07b329 pkg/kgo: elide broker log line if context is canceled
Additional commits viewable in compare view

Updates github.com/twmb/franz-go/pkg/kadm from 1.9.0 to 1.11.0

Changelog

Sourced from github.com/twmb/franz-go/pkg/kadm's changelog.

v1.11.0

This is a small release containing two minor features and a few behavior improvements. The MarkedOffsets function allows for manually committing marked offsets if you override OnPartitionsRevoked, and UnsafeAbortBufferedRecords allows for forcefully dropping anything being produced (albeit with documented caveats and downsides)

The client now guards against a broker that advertises FetchRequest v13+ (which only uses TopicIDs) but does not actually return / use TopicIDs. If you have an old IBP configured, the broker will not use TopicIDs even if the broker indicates it should. The client will now pin fetching to a max version of 12 if a topic has no TopicID.

The client now sets a record's Topic field earlier to DefaultProduceTopic, which allows the Topic field to be known present (or known non-present) in the OnRecordBuffered hook.

Lastly, we now universally use Go 1.19 atomic types if compiled with 1.19+. Go uses compiler intrinsics to ensure proper int64 / uint64 alignment within structs for the atomic types; Go does not ensure plain int64 / uint64 are properly aligned. A lot of work previously went into ensuring alignment and having a GitHub workflow that ran go vet on qemu armv7 emulation, but apparently that was not comprehensive enough. Now, if you use a 32 bit arch, it is recommended to just compile with 1.19+.

d1b6897 feature kgo: add UnsafeAbortBufferedRecords

d0c42ad improvement kgo source: do not use fetch topic IDs if the broker returns no ID

cc3a355 and a2c4bad improvement kgo: universally switch to 1.19's atomics if on Go 1.19+

66e626f producer: set Record.Topic earlier

3186e61 feature kgo: add MarkedOffsets function

v1.10.4

This patch release fixes two bugs introduced with v1.10.0. These bugs are not encountered in when using the client to simply consume or produce. Only admin usages of the client may encounter the bugs this patch is fixing.

v1.10.0 introduced support for batch offset fetching or coordinator finding. These changes introduced a bug where empty coordinator keys (i.e., group names or transactional IDs) would be stripped from requests, and then a field in a nil pointer could be accessed and panic the program. These changes also introduced a bug that did not properly mirror one field for batched FindCoordinator requests.

ca67da4 bugfix kgo: fix batch coordinator fetching

c6f7f9a bugfix kgo: allow empty groups when finding coordinator / fetching offsets

... (truncated)

Commits

a90ab8a Merge pull request #306 from twmb/1.11.0
80073c5 CHANGELOG: fixup v1.11.0 bullets
7a1484d CHANGELOG: note incoming v1.11.0
9beb8cd Merge pull request #305 from twmb/1.11.0
e81c929 Merge pull request #303 from twmb/no-kip-792-proto
09d8b77 Revert "kmsg: commit new ConsumerMemberMetadata for KIP-792"
476f540 Merge pull request #302 from twmb/unsafe_abort
d1b6897 kgo: add UnsafeAbortBufferedRecords
b942117 Merge pull request #280 from twmb/marked_offsets
e1206c0 Merge pull request #301 from twmb/minor
Additional commits viewable in compare view

You can trigger a rebase of this PR by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
@dependabot ignore <dependency name> major version will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself)
@dependabot ignore <dependency name> minor version will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself)
@dependabot ignore <dependency name> will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself)
@dependabot unignore <dependency name> will remove all of the ignore conditions of the specified dependency
@dependabot unignore <dependency name> <ignore condition> will remove the ignore condition of the specified dependency and ignore conditions

Note Automatic rebases have been disabled on this pull request as it has been open for over 30 days.

Jan 22 '24 20:01 dependabot[bot]

Bloop Bleep... Dogbot Here

Regression Detector Results

Run ID: 0453b0d9-a2d8-4ba0-acb9-3d6818da4933 Baseline: f1a15cbfb4f13a337d6d61101a644dfb8053a6d2 Comparison: 146d798901a69d68f13d14852858f7b97162bc0a

Performance changes are noted in the perf column of each table:

✅ = significantly better comparison variant performance
❌ = significantly worse comparison variant performance
➖ = no significant change in performance

No significant changes in experiment optimization goals

Confidence level: 90.00% Effect size tolerance: |Δ mean %| ≥ 5.00%

There were no significant changes in experiment optimization goals at this confidence level and effect size tolerance.

Experiments ignored for regressions

Regressions in experiments with settings containing erratic: true are ignored.

perf	experiment	goal	Δ mean %	Δ mean % CI
➖	file_to_blackhole	% cpu utilization	+0.08	[-6.50, +6.66]

Fine details of change detection per experiment

perf	experiment	goal	Δ mean %	Δ mean % CI
➖	file_tree	memory utilization	+0.40	[+0.32, +0.48]
➖	basic_py_check	% cpu utilization	+0.22	[-2.10, +2.54]
➖	file_to_blackhole	% cpu utilization	+0.08	[-6.50, +6.66]
➖	trace_agent_json	ingress throughput	+0.01	[-0.03, +0.04]
➖	uds_dogstatsd_to_api	ingress throughput	+0.00	[-0.00, +0.00]
➖	tcp_dd_logs_filter_exclude	ingress throughput	-0.00	[-0.00, +0.00]
➖	trace_agent_msgpack	ingress throughput	-0.01	[-0.02, -0.01]
➖	process_agent_standard_check_with_stats	memory utilization	-0.11	[-0.15, -0.08]
➖	idle	memory utilization	-0.13	[-0.17, -0.09]
➖	process_agent_real_time_mode	memory utilization	-0.31	[-0.35, -0.27]
➖	tcp_syslog_to_blackhole	ingress throughput	-0.43	[-0.48, -0.37]
➖	otel_to_otel_logs	ingress throughput	-0.53	[-1.18, +0.12]
➖	process_agent_standard_check	memory utilization	-0.55	[-0.59, -0.51]
➖	uds_dogstatsd_to_api_cpu	% cpu utilization	-1.89	[-3.30, -0.47]

Explanation

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
Its configuration does not mark it "erratic".

Jan 22 '24 22:01 pr-commenter[bot]

Test changes on VM

Use this command from test-infra-definitions to manually test this PR changes on a VM:

inv create-vm --pipeline-id=30331302 --os-family=ubuntu

Mar 13 '24 19:03 pr-commenter[bot]