[QUESTION] What are the requirements for the C extensions ?

[vmaurin]

It seems by default aiokafka tries to load C extensions for core/cpu intensive operations. I have seen various blocks initializing it like

    try:
        from ._crecords import (
            DefaultRecordBatchBuilder as _DefaultRecordBatchBuilderCython,
            DefaultRecordMetadata as _DefaultRecordMetadataCython,
            DefaultRecordBatch as _DefaultRecordBatchCython,
            DefaultRecord as _DefaultRecordCython,
        )
        DefaultRecordBatchBuilder = _DefaultRecordBatchBuilderCython
        DefaultRecordMetadata = _DefaultRecordMetadataCython
        DefaultRecordBatch = _DefaultRecordBatchCython
        DefaultRecord = _DefaultRecordCython
    except ImportError:  # pragma: no cover
        DefaultRecordBatchBuilder = _DefaultRecordBatchBuilderPy
        DefaultRecordMetadata = _DefaultRecordMetadataPy
        DefaultRecordBatch = _DefaultRecordBatchPy
        DefaultRecord = _DefaultRecordPy

My issue is that it is not clear from the documentation what could lead to the extension not being loaded (i.e the except block) and it could be a bit silent. Like do I need specific libs to be installed on the OS for the extension to work ?

Then according the answer, do you think it could be useful to log a warning when we hit an ImportError like "Unable to load C extensions, fallback to less performant Python bindings" ?

[ods]

All binary wheels contain pre-build extensions, but you can set AIOKAFKA_NO_EXTENSIONS=1 environment variable to disable them. Also you can import aiokafka directly from source, when extensions are not built.

[vmaurin]

@ods Thank you for your anwser

I actually do want to use them, I was wondering how to be sure it is used/loaded properly :

• do they depend on a dynamic linked lib (like libgzip or similar) ?
• should a message be displayed in the except block to inform that something was wrong ?