Unable to install confluent-kafka library from source in docker container

[sngit1234]

trafficstars

Description

Getting Error when installing confluent-kafka from source for GSSAPI support #5 9.889 2673 | error = rd_kafka_sasl_set_credentials(self->rk, username, password); #5 9.889 | ^ #5 9.889 error: command 'gcc' failed with exit status 1

Docker Image build successfully if I just do pip install confluent-kafka

#5 9.889 /tmp/pip-install-yhu3z7rm/confluent-kafka_3ce49d6053ba45df9049ca9e51ff7fdc/src/confluent_kafka/src/confluent_kafka.c:1466:46: warning: implicit declaration of function ‘rd_kafka_Node_host’; did you mean ‘rd_kafka_conf_get’? [-Wimplicit-function-declaration] #5 9.889 1466 | cfl_PyDict_SetString(kwargs, "host", rd_kafka_Node_host(c_node)); #5 9.889 | ^~~~~~~~~~~~~~~~~~ #5 9.889 | rd_kafka_conf_get #5 9.889 /tmp/pip-install-yhu3z7rm/confluent-kafka_3ce49d6053ba45df9049ca9e51ff7fdc/src/confluent_kafka/src/confluent_kafka.c:1466:46: warning: passing argument 3 of ‘cfl_PyDict_SetString’ makes pointer from integer without a cast [-Wint-conversion] #5 9.889 1466 | cfl_PyDict_SetString(kwargs, "host", rd_kafka_Node_host(c_node)); #5 9.889 | ^~~~~~~~~~~~~~~~~~~~~~~~~~ #5 9.889 | | #5 9.889 | int #5 9.889 In file included from /tmp/pip-install-yhu3z7rm/confluent-kafka_3ce49d6053ba45df9049ca9e51ff7fdc/src/confluent_kafka/src/confluent_kafka.c:17: #5 9.889 /tmp/pip-install-yhu3z7rm/confluent-kafka_3ce49d6053ba45df9049ca9e51ff7fdc/src/confluent_kafka/src/confluent_kafka.h:328:74: note: expected ‘const char *’ but argument is of type ‘int’ #5 9.889 328 | void cfl_PyDict_SetString (PyObject *dict, const char *name, const char *val); #5 9.889 | ~~~~~~~~~~~~^~~ #5 9.889 /tmp/pip-install-yhu3z7rm/confluent-kafka_3ce49d6053ba45df9049ca9e51ff7fdc/src/confluent_kafka/src/confluent_kafka.c: In function ‘set_sasl_credentials’: #5 9.889 /tmp/pip-install-yhu3z7rm/confluent-kafka_3ce49d6053ba45df9049ca9e51ff7fdc/src/confluent_kafka/src/confluent_kafka.c:2673:17: warning: implicit declaration of function ‘rd_kafka_sasl_set_credentials’; did you mean ‘rd_kafka_conf_set_events’? [-Wimplicit-function-declaration] #5 9.889 2673 | error = rd_kafka_sasl_set_credentials(self->rk, username, password); #5 9.889 | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #5 9.889 | rd_kafka_conf_set_events #5 9.889 /tmp/pip-install-yhu3z7rm/confluent-kafka_3ce49d6053ba45df9049ca9e51ff7fdc/src/confluent_kafka/src/confluent_kafka.c:2673:15: warning: assignment to ‘rd_kafka_error_t *’ {aka ‘struct rd_kafka_error_s *’} from ‘int’ makes pointer from integer without a cast [-Wint-conversion] #5 9.889 2673 | error = rd_kafka_sasl_set_credentials(self->rk, username, password); #5 9.889 | ^ #5 9.889 error: command 'gcc' failed with exit status 1 #5 9.889 [end of output] #5 9.889 #5 9.889 note: This error originates from a subprocess, and is likely not a problem with pip. #5 9.897 error: legacy-install-failure #5 9.897 #5 9.897 × Encountered error while trying to install package. #5 9.897 ╰─> confluent-kafka #5 9.897 #5 9.897 note: This is an issue with the package mentioned above, not pip. #5 9.897 hint: See above for output from the failure. #5 11.08 WARNING: You are using pip version 22.0.4; however, version 23.1.2 is available. #5 11.08 You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command. #5 ERROR: executor failed running [/bin/sh -c pip3 install --no-binary :all: confluent-kafka]: exit code: 1

[3/3] RUN pip3 install --no-binary :all: confluent-kafka:

---

executor failed running [/bin/sh -c pip3 install --no-binary :all: confluent-kafka]: exit code: 1

How to reproduce

Here is the docker file used to build mage

FROM python:3.8.13-slim-bullseye

RUN apt-get update &&
apt-get install -y --no-install-recommends gcc git libssl-dev g++ make librdkafka-dev

RUN pip3 install --no-binary :all: confluent-kafka

Checklist

Please provide the following information:

• [ ] confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): default
• [ ] Apache Kafka broker version:
• [ ] Client configuration: {...}
• [ ] Operating system: debian
• [ ] Provide client logs (with 'debug': '..' as necessary)
• [ ] Provide broker log excerpts
• [ ] Critical issue

[sivankumar86]

Have you tried this ?

FROM python:3.9.11-alpine3.15 


# upgrade pip to the latest version
RUN apk --no-cache upgrade \
    && pip install --upgrade pip \
    && apk --no-cache add tzdata build-base gcc libc-dev g++ make git bash

RUN git clone https://github.com/edenhill/librdkafka.git && cd librdkafka \
    && git checkout tags/v2.2.0 && ./configure --clean \
    && ./configure --prefix /usr/local \
    && make && make install
RUN pip install confluent-kafka==2.2.0

[nhaq-confluent]

@sngit1234 can you confirm if the previous comment helps and if this issue is still required to be open?