Bumps the franz-go group with 2 updates: github.com/twmb/franz-go and github.com/twmb/franz-go/pkg/kadm.

Updates github.com/twmb/franz-go from 1.14.3 to 1.16.1

Changelog

Sourced from github.com/twmb/franz-go's changelog.

v1.16.1

This patch release fixes one bug and un-deprecates SaramaHasher.

SaramaHasher, while not identical to Sarama's partitioner, actually is identical to some other partitioners in the Kafka client ecosystem. So, the old function is now un-deprecated, but the documentation correctly points you to SaramaCompatHasher and mentions why you may still want to use SaramaHasher.

For the bug: if you tried using CommitOffsetsSync during a group rebalance, and you canceled your context while the group was still rebalancing, then CommitOffsetsSync would enter a deadlock and never return. That has been fixed.

cd65d77 and 99d6dfb kgo: fix bug

d40ac19 kgo: un-deprecate SaramaHasher and add docs explaining why

v1.16.0

This release contains a few minor APIs and internal improvements and fixes two minor bugs.

One new API that is introduced also fixes a bug. API-wise, the SaramaHasher was actually not a 1:1 compatible hasher. The logic was identical, but there was a rounding error because Sarama uses int32 module arithmetic, whereas kgo used int (which is likely int64) which caused a different hash result. A new SaramaCompatHasher has been introduced and the old SaramaHasher has been deprecated.

The other bugfix is that OptValue on the kgo.Logger option panicked if you were not using a logger. That has been fixed.

The only other APIs that are introduced are in the kversions package; they are minor, see the commit list below.

If you issue a sharded request and any of the responses has a retryable error in the response, this is no-longer returned as a top-level shard error. The shard error is now nil, and you can properly inspect the response fully.

Lastly (besides other internal minor improvements not worth mentioning), metadata fetches can now inject fake fetches if the metadata response has topic or partition load errors. This is unconditionally true for non-retryable errors. If you use KeepRetryableFetchErrors, you can now also see when metadata fetching is showing unknown topic errors or other retryable errors.

a2340eb improvement pkg/kgo: inject fake fetches on metadata load errors

d07efd9 feature kversion: add VersionStrings, FromString, V3_6_0

8d30de0 bugfix pkg/kgo: fix OptValue with no logger set

012cd7c improvement kgo: do not return response ErrorCode's as shard errors

... (truncated)

Commits

c5207aa Merge pull request #674 from twmb/patches
e08d276 CHANGELOG: note further patch commit
99d6dfb kgo: further fix for cd65d77d05f71ca51d81d247312d0d465fdb76c8
41f0269 Merge pull request #673 from twmb/patch
20867cd CHANGELOG: note incoming v1.16.1
cd65d77 kgo: fix bug
d40ac19 kgo: un-deprecate SaramaHasher and add docs explaining why
a2d69ce Merge pull request #666 from twmb/kadm
0b89158 kadm: add DeleteTopic,DeleteGroup,Error
c1a3c3d Merge pull request #664 from twmb/changelog-v1.16
Additional commits viewable in compare view

Updates github.com/twmb/franz-go/pkg/kadm from 1.9.0 to 1.11.0

Changelog

Sourced from github.com/twmb/franz-go/pkg/kadm's changelog.

v1.11.0

This is a small release containing two minor features and a few behavior improvements. The MarkedOffsets function allows for manually committing marked offsets if you override OnPartitionsRevoked, and UnsafeAbortBufferedRecords allows for forcefully dropping anything being produced (albeit with documented caveats and downsides)

The client now guards against a broker that advertises FetchRequest v13+ (which only uses TopicIDs) but does not actually return / use TopicIDs. If you have an old IBP configured, the broker will not use TopicIDs even if the broker indicates it should. The client will now pin fetching to a max version of 12 if a topic has no TopicID.

The client now sets a record's Topic field earlier to DefaultProduceTopic, which allows the Topic field to be known present (or known non-present) in the OnRecordBuffered hook.

Lastly, we now universally use Go 1.19 atomic types if compiled with 1.19+. Go uses compiler intrinsics to ensure proper int64 / uint64 alignment within structs for the atomic types; Go does not ensure plain int64 / uint64 are properly aligned. A lot of work previously went into ensuring alignment and having a GitHub workflow that ran go vet on qemu armv7 emulation, but apparently that was not comprehensive enough. Now, if you use a 32 bit arch, it is recommended to just compile with 1.19+.

d1b6897 feature kgo: add UnsafeAbortBufferedRecords

d0c42ad improvement kgo source: do not use fetch topic IDs if the broker returns no ID

cc3a355 and a2c4bad improvement kgo: universally switch to 1.19's atomics if on Go 1.19+

66e626f producer: set Record.Topic earlier

3186e61 feature kgo: add MarkedOffsets function

v1.10.4

This patch release fixes two bugs introduced with v1.10.0. These bugs are not encountered in when using the client to simply consume or produce. Only admin usages of the client may encounter the bugs this patch is fixing.

v1.10.0 introduced support for batch offset fetching or coordinator finding. These changes introduced a bug where empty coordinator keys (i.e., group names or transactional IDs) would be stripped from requests, and then a field in a nil pointer could be accessed and panic the program. These changes also introduced a bug that did not properly mirror one field for batched FindCoordinator requests.

ca67da4 bugfix kgo: fix batch coordinator fetching

c6f7f9a bugfix kgo: allow empty groups when finding coordinator / fetching offsets

... (truncated)

Commits

a90ab8a Merge pull request #306 from twmb/1.11.0
80073c5 CHANGELOG: fixup v1.11.0 bullets
7a1484d CHANGELOG: note incoming v1.11.0
9beb8cd Merge pull request #305 from twmb/1.11.0
e81c929 Merge pull request #303 from twmb/no-kip-792-proto
09d8b77 Revert "kmsg: commit new ConsumerMemberMetadata for KIP-792"
476f540 Merge pull request #302 from twmb/unsafe_abort
d1b6897 kgo: add UnsafeAbortBufferedRecords
b942117 Merge pull request #280 from twmb/marked_offsets
e1206c0 Merge pull request #301 from twmb/minor
Additional commits viewable in compare view

You can trigger a rebase of this PR by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
@dependabot ignore <dependency name> major version will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself)
@dependabot ignore <dependency name> minor version will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself)
@dependabot ignore <dependency name> will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself)
@dependabot unignore <dependency name> will remove all of the ignore conditions of the specified dependency
@dependabot unignore <dependency name> <ignore condition> will remove the ignore condition of the specified dependency and ignore conditions

Note Automatic rebases have been disabled on this pull request as it has been open for over 30 days.

Feb 08 '24 15:02 dependabot[bot]

Bloop Bleep... Dogbot Here

Regression Detector Results

Run ID: 01d452c1-f53d-41f0-bce9-d0080fff7bb9 Baseline: f1a15cbfb4f13a337d6d61101a644dfb8053a6d2 Comparison: 8aea357925133a7cec308831a7334cc3a1450a35

Performance changes are noted in the perf column of each table:

✅ = significantly better comparison variant performance
❌ = significantly worse comparison variant performance
➖ = no significant change in performance

No significant changes in experiment optimization goals

Confidence level: 90.00% Effect size tolerance: |Δ mean %| ≥ 5.00%

There were no significant changes in experiment optimization goals at this confidence level and effect size tolerance.

Experiments ignored for regressions

Regressions in experiments with settings containing erratic: true are ignored.

perf	experiment	goal	Δ mean %	Δ mean % CI
➖	file_to_blackhole	% cpu utilization	+1.50	[-5.11, +8.10]

Fine details of change detection per experiment

perf	experiment	goal	Δ mean %	Δ mean % CI
➖	basic_py_check	% cpu utilization	+1.90	[-0.36, +4.17]
➖	file_to_blackhole	% cpu utilization	+1.50	[-5.11, +8.10]
➖	process_agent_standard_check_with_stats	memory utilization	+0.59	[+0.55, +0.63]
➖	process_agent_real_time_mode	memory utilization	+0.32	[+0.27, +0.37]
➖	trace_agent_json	ingress throughput	+0.04	[+0.01, +0.07]
➖	file_tree	memory utilization	+0.03	[-0.09, +0.14]
➖	tcp_dd_logs_filter_exclude	ingress throughput	+0.00	[-0.00, +0.00]
➖	uds_dogstatsd_to_api	ingress throughput	-0.00	[-0.00, +0.00]
➖	trace_agent_msgpack	ingress throughput	-0.02	[-0.03, -0.01]
➖	tcp_syslog_to_blackhole	ingress throughput	-0.03	[-0.09, +0.03]
➖	process_agent_standard_check	memory utilization	-0.07	[-0.11, -0.03]
➖	idle	memory utilization	-0.07	[-0.11, -0.03]
➖	otel_to_otel_logs	ingress throughput	-0.45	[-1.09, +0.19]
➖	uds_dogstatsd_to_api_cpu	% cpu utilization	-2.10	[-3.52, -0.68]

Explanation

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
Its configuration does not mark it "erratic".

Feb 08 '24 17:02 pr-commenter[bot]

Test changes on VM

Use this command from test-infra-definitions to manually test this PR changes on a VM:

inv create-vm --pipeline-id=30331787 --os-family=ubuntu

Mar 13 '24 19:03 pr-commenter[bot]