Andrei Zhabinski

According to [`rdkafka.h`](https://github.com/edenhill/librdkafka/blob/master/src/rdkafka.h#L2975-L3002) (which is one of the primary sources of documentation btw): > `@remark` An application should make sure to call poll() at regular > intervals to serve any...

I've just merged #13 with polling in the `@async`-created task (i.e. in the same thread). If I'm not missing anything, rebasing this PR on master now should be sufficient to...

```julia p.client.conf["message.max.bytes"] = 36423360 p.client.conf["max.message.bytes"] = 36423360 ``` Ouch, this is not supposed to work :fearful: `conf` is passed to the `KafkaClient` constructor and used to instantiate C-level client object....

librdkafka should be thread-safe by itself, but we may corrupt memory e.g. while destroying the allocated resources. Can you post a minimal reproducible example (the producer side should be enough)?...

I cannot reproduce the crash with a freshly installed RDKafka.jl on Linux. Messages of size > 1Mb indeed are not delivered, but it might be related to other settings (e.g....

If you continue sending messages endlessly (so producer is never "done"), do you see the same error?

Does the script work fine with messages less than 1Mb? Large messages may be blocked by many settings on both - client and server side, so one direction to go...

Just checked it with the latest transformers, latest JAX and latest CUDA on Nvidia H100, but the problem persists. Can somebody try it and confirm/reject that it is reproducible? On...

Could you please re-build Spark.jl and post the log here? ```julia ] build Spark ```

I don't see anything strange in your log and can't reproduce it, so let's start with a couple of tests. Check if JavaCall works fine: ```julia # Julia REPL using...