librdkafka icon indicating copy to clipboard operation
librdkafka copied to clipboard

rd_kafka_produceva: double free headers on message sending error

Open blindspotbounty opened this issue 1 year ago • 2 comments

Description

Accidentally, we found a problem related to headers double free in rd_kafka_produceva method:

=================================================================
==49557==ERROR: AddressSanitizer: heap-use-after-free on address 0x0001178e9dd8 at pc 0x00010f6813b4 bp 0x00016b719000 sp 0x00016b718ff8
READ of size 8 at 0x0001178e9dd8 thread T18
#0 0x10f6813b0 in rd_list_destroy_elems rdlist.c:276
    #1 0x10f681828 in rd_list_destroy rdlist.c:300
    #2 0x10f2fb2d0 in rd_kafka_headers_destroy rdkafka_header.c:37
    #3 0x10f465414 in rd_kafka_produceva rdkafka_msg.c:521
    #4 0x10fbcc5dc in RDKafkaClient._produceVariadic(topicHandle:partition:messageFlags:key:value:opaque:cHeaders:) RDKafkaClient.swift:218
    #5 0x10fbc8910 in closure #1 in closure #1 in closure #1 in RDKafkaClient.produce<A, B>(message:newMessageID:topicConfiguration:topicHandles:) RDKafkaClient.swift:137
    #6 0x10fbe5188 in partial apply for closure #1 in closure #1 in closure #1 in RDKafkaClient.produce<A, B>(message:newMessageID:topicConfiguration:topicHandles:) <compiler-generated>
    ...
    
0x0001178e9dd8 is located 8 bytes inside of 48-byte region [0x0001178e9dd0,0x0001178e9e00)
freed by thread T18 here:
#0 0x105133380 in wrap_free+0x98 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x53380)
    #1 0x10f2fb2fc in rd_free rd.h:151
    #2 0x10f2fb2d8 in rd_kafka_headers_destroy rdkafka_header.c:38
    #3 0x10f460c04 in rd_kafka_msg_destroy rdkafka_msg.c:118
    #4 0x10f4652ec in rd_kafka_produceva rdkafka_msg.c:505
...

previously allocated by thread T18 here:
#0 0x105133244 in wrap_malloc+0x94 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x53244)
    #1 0x10f2fb3b4 in rd_malloc rd.h:139
    #2 0x10f2fb330 in rd_kafka_headers_new rdkafka_header.c:44
    #3 0x10f4646f4 in rd_kafka_produceva rdkafka_msg.c:431

How to reproduce

I was reproducing it with swift-kafka-client wrapping client within issue https://github.com/swift-server/swift-kafka-client/issues/150

git clone --recursive https://github.com/swift-server/swift-kafka-client
git apply diff // (from issue)
swift test --sanitize=address --filter testProduceAndConsumeWithMessageHeaders

Checklist

IMPORTANT: We will close issues where the checklist has not been completed.

Please provide the following information:

  • [x] librdkafka version (release number or git tag): v2.3.0
  • [x] Apache Kafka version: N/A
  • [x] librdkafka client configuration: any invalid broker address
  • [x] Operating system: ubuntu 22.04 lts
  • [x] Provide logs (with debug=.. as necessary) from librdkafka: ASAN logs instead
  • [x] Provide broker log excerpts: N/A
  • Not a critical issue

blindspotbounty avatar Feb 28 '24 15:02 blindspotbounty

Yeah, the hdrs variable should be set to NULL before rd_kafka_msg_destroy if it corresponds to the message headers, otherwise it's freed a second time here

emasab avatar Mar 27 '24 10:03 emasab

Yes, exactly. Btw: fixed comment as requested in PR.

blindspotbounty avatar Apr 08 '24 12:04 blindspotbounty