[bitnami/redis-cluster] Module RSCoordinator (module-oss.so) is not loading

[dilipk8]

Name and Version

bitnami/redis-cluster v9.1.3

What architecture are you using?

amd64

What steps will reproduce the bug?

I have a bitnami/redis-cluster ( version 9.1.3) helm chart deployed on EKS. The Redistimeseries,graph,search modules are loading fine. We want distributed search across the cluster to work. I have an issue with the rscoordinator module and this is not loading. i built the module using the instruction from here - https://redis.io/docs/interact/search-and-query/deprecated/development/

Error message -

Module /redismodules/module-oss.so failed to load: libcrypto.so.3: cannot open shared object file: No such file or directory I have double-checked and exec'ed into the pod and verified that module-oss.so is present in the redismodules directory.

Is there any workaround to load and fix this issue ? And would also like to confirm if distributed cluster-wide search would work if this fixed ?

Are you using any custom parameters or values?

No response

What is the expected behavior?

RSCoordinator module should load just like other redis module without any errors/issues.

What do you see instead?

Error message in the logs

Module /redismodules/module-oss.so failed to load: libcrypto.so.3: cannot open shared object file: No such file or directory

Additional information

No response

[github-actions[bot]]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

[dilipk8]

Hello, Any update on the issue ?

[github-actions[bot]]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

[aoterolorenzo]

Hi @dilipk8,

So sorry for the delay, I just missed completely this support case. Could you provide any details to reproduce this issue from our side? Seems a pretty complex scenario to easily do so

[github-actions[bot]]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

[github-actions[bot]]

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.

[dilipk8]

Hi @aoterolorenzo,

Steps - 

1. Run - git clone --recursive https://github.com/RediSearch/RediSearch.git
2. cd RediSearch
3. ./sbin/setup  ( Run as root )
4. sudo make build COORD=oss

Above commands would generate a module-oss.so file. This file is then used in the load module command, in the redis-cluster setup. I have created a custom redis-stack server image (container) that has this module and am trying to load this module, just like other redis modules

eg: loadmodule /redismodules/module-oss.so

[aoterolorenzo]

Hi @dilipk8,

Seems redis-cluster misses the openssl dependency, and that dependency is needed for the module-oss you try to install. There is some discussion here with what's the best solution, since the idea of adding all module dependencies to the redis-cluster image doesn't sound fine, and we would be extending the container's surface attack (and probably several modules imply several more dependencies, so it is not something maintainable). Let me bring this discussion to the team, and I will reach you back here asap.

[aoterolorenzo]

Hi again @dilipk8,

In this case, as I was mentioning, we don't contemplate the option of adding extra dependencies to our bitnami/redis-cluster image because of a single module, so in this case we would recommend you to create your custom extended image using our bitnami/redis-cluster as a base image, and installing any dependencies you need (in this case I think openssl would be enough) and even preparing the modules there. Then, you can use it in our chart, changing the image values to set your own instead of ours.

Hope this helps.

PS: Here is an example on how to extend a Bitnami docker image (in this case the phpbb image): https://github.com/bitnami/containers/tree/main/bitnami/phpbb#extend-this-image

[dilipk8]

Hi @aoterolorenzo , Thanks for the reply. I will try to use the base image and extend it and probably also include any other packages. Thanks for the link.

[github-actions[bot]]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

[github-actions[bot]]

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.

[svabra]

@aoterolorenzo Did you manage to get this running?

[xtianus79]

i am going to try now. i have been successful on a custom image build. I will update.

[svabra]

@xtianus79 any compilation results you could share?

[xtianus79]

@svabra and @aoterolorenzo Yes lot's to share. I spent all weekend on this.

i forked the repo and fixed a ton of errors. https://github.com/xtianus79/RediSearch/tree/master

this is a sub module that I had to fix too for reference (don't need it for build) https://github.com/xtianus79/RedisLabsModules-readies/tree/master

here is my docker file. I haven't deployed this to git as of yet.

But, alas :( don't get too excited because while I can build the module ( this is just for module-oss and module-enterprise rlec) git submodule update --init --recursive cd coord cmake -DCOORD_TYPE=oss . make

it doesn't actually load the module. I've added all dependencies needed in the custom image below. here is the error message I get

1:M 08 Apr 2024 05:51:54.639 # Module /opt/bitnami/redis/etc/module-oss.so initialization failed. Module not loaded
1:M 08 Apr 2024 05:51:54.639 # Can't load module from /opt/bitnami/redis/etc/module-oss.so: server aborting

At this point I am a little lost.

# Stage 1: Build environment for RedisJSON with an appropriate Python version
FROM python:3.9 AS redisjson-build-env

# Install system dependencies and clean up in one layer
RUN apt-get update && \
    for i in $(seq 1 3); do \
        apt-get install -y --no-install-recommends build-essential git curl && break || sleep 15; \
    done && \
    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/* 

# Set PATH for Rust
ENV PATH="/root/.cargo/bin:${PATH}"

# Clone and build RedisJSON
WORKDIR /build/redisjson
RUN git clone --recursive https://github.com/RedisJSON/RedisJSON.git . && \
    python3 -m venv venv && \
    . venv/bin/activate && \
    pip install --upgrade pip setuptools wheel && \
    ./sbin/setup && \
    cargo build --release

# Verify the build artifacts for librejson.so
RUN echo "Verifying librejson.so build artifacts:" && \
    ls -la /build/redisjson/target/release/ && \
    echo "librejson.so build completed successfully if listed above."

# Stage 2: Build environment for RediSearch
FROM debian:bullseye-slim AS redisearch-build-env

# Install Python, build tools, CMake, and git
RUN apt-get update && \
    for i in $(seq 1 3); do \
        apt-get install -y --no-install-recommends \
        python3 python3-pip python3-venv build-essential libboost-all-dev cmake git && break || sleep 15; \
    done && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Clone and build RedisSearch
WORKDIR /build/redisearch
RUN git clone --recursive --branch v2.8.12 https://github.com/RediSearch/RediSearch.git . && \
    python3 -m venv venv && \
    . venv/bin/activate && \
    pip install --upgrade pip setuptools wheel && \
    pip install conan && \
    make

# After building, list the contents of the expected output directory
# Adjust the path below if the build artifacts are placed in a different location
# After building, list the contents of the build directory to confirm the existence of redisearch.so
RUN ls -la /build/redisearch/bin/linux-x64-release/search && \
    echo "redisearch.so build artifacts:" && \
    ls -la /build/redisearch/bin/linux-x64-release/search/redisearch.so && \
    echo "redisearch.so build completed successfully."

# Copy the libcrypto.so.1.1 and libssl.so.1.1 libraries from the Debian base image
RUN cp /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 /build/redisearch/ && \
    cp /usr/lib/x86_64-linux-gnu/libssl.so.1.1 /build/redisearch/

# Stage 3: Build environment for RediCoordination
FROM debian:bullseye-slim AS rediscoordination-build-env

# Install Python, build tools, CMake, git, OpenSSL development libraries, and other dependencies
RUN apt-get update && \
    for i in $(seq 1 3); do \
        apt-get install -y --no-install-recommends \
        python3 python3-pip python3-venv build-essential libboost-all-dev cmake git wget libssl-dev libuv1-dev && break || sleep 15; \
    done && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Clone your forked RediSearch repository
WORKDIR /build/redisearch
RUN git clone --recursive https://github.com/xtianus79/RediSearch.git .

# Install Boost 1.81.0
RUN apt-get update && \
    for i in $(seq 1 3); do \
        apt-get install -y --no-install-recommends wget && break || sleep 15; \
    done && \
    mkdir /boost && \
    cd /boost && \
    wget https://boostorg.jfrog.io/artifactory/main/release/1.81.0/source/boost_1_81_0.tar.bz2 && \
    tar -xvf boost_1_81_0.tar.bz2 && \
    cd boost_1_81_0/ && \
    ./bootstrap.sh && \
    ./b2 install && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Set the LD_LIBRARY_PATH environment variable
ENV LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH"

# Set up the build environment
RUN python3 -m venv venv && \
    . venv/bin/activate && \
    pip install --upgrade pip setuptools wheel && \
    pip install conan

# Build the project
WORKDIR /build/redisearch/coord
RUN mkdir build && \
    cd build && \
    cmake -DCOORD_TYPE=oss .. && \
    make

# After building, list the contents of the build directory to confirm the existence of module-oss.so
RUN ls -la /build/redisearch/coord/build && \
    echo "module-oss.so build artifacts:" && \
    ls -la /build/redisearch/coord/build/module-oss.so && \
    echo "module-oss.so build completed successfully."

# Stage 4: Build environment for RedisAI with GPU support
## cut for brevity 

# Copyright VMware, Inc.
# SPDX-License-Identifier: APACHE-2.0
# Stage 5: Final image

FROM docker.io/bitnami/minideb:bookworm

ARG TARGETARCH

LABEL com.vmware.cp.artifact.flavor="sha256:c50c90cfd9d12b445b011e6ad529f1ad3daea45c26d20b00732fae3cd71f6a83" \
      org.opencontainers.image.base.name="docker.io/bitnami/minideb:bookworm" \
      org.opencontainers.image.created="2024-03-31T19:42:32Z" \
      org.opencontainers.image.description="Application packaged by VMware, Inc" \
      org.opencontainers.image.licenses="Apache-2.0" \
      org.opencontainers.image.ref.name="7.2.4-debian-12-r11" \
      org.opencontainers.image.title="redis-cluster" \
      org.opencontainers.image.vendor="VMware, Inc." \
      org.opencontainers.image.version="7.2.4"

ENV HOME="/" \
    OS_ARCH="${TARGETARCH:-amd64}" \
    OS_FLAVOUR="debian-12" \
    OS_NAME="linux"

COPY prebuildfs /
SHELL ["/bin/bash", "-o", "errexit", "-o", "nounset", "-o", "pipefail", "-c"]
# Install required system packages and dependencies
USER root
RUN apt-get update && \
    apt-get install -y ca-certificates curl libgomp1 libssl3 procps libuv1 && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Copy compiled modules from the build stages
# COPY --from=redisjson-build-env /build/redisjson/target/release/librejson.so /usr/lib/rejson.so
# COPY --from=redisearch-build-env /build/redisearch/bin/linux-x64-release/search/redisearch.so /usr/lib/redisearch.so

COPY --from=redisjson-build-env /build/redisjson/target/release/librejson.so /opt/bitnami/redis/etc/rejson.so
COPY --from=redisearch-build-env /build/redisearch/bin/linux-x64-release/search/redisearch.so /opt/bitnami/redis/etc/redisearch.so
COPY --from=rediscoordination-build-env /build/redisearch/coord/build/module-oss.so /opt/bitnami/redis/etc/module-oss.so

# Copy the libcrypto.so.1.1 and libssl.so.1.1 libraries from the redisearch-build-env stage
COPY --from=redisearch-build-env /build/redisearch/libcrypto.so.1.1 /usr/lib/x86_64-linux-gnu/
COPY --from=redisearch-build-env /build/redisearch/libssl.so.1.1 /usr/lib/x86_64-linux-gnu/

# Ensure the library is found
RUN ldconfig

RUN mkdir -p /tmp/bitnami/pkg/cache/ ; cd /tmp/bitnami/pkg/cache/ ; \
    COMPONENTS=( \
      "wait-for-port-1.0.7-10-linux-${OS_ARCH}-debian-12" \
      "redis-7.2.4-3-linux-${OS_ARCH}-debian-12" \
    ) ; \
    for COMPONENT in "${COMPONENTS[@]}"; do \
      if [ ! -f "${COMPONENT}.tar.gz" ]; then \
        curl -SsLf "https://downloads.bitnami.com/files/stacksmith/${COMPONENT}.tar.gz" -O ; \
        curl -SsLf "https://downloads.bitnami.com/files/stacksmith/${COMPONENT}.tar.gz.sha256" -O ; \
      fi ; \
      sha256sum -c "${COMPONENT}.tar.gz.sha256" ; \
      tar -zxf "${COMPONENT}.tar.gz" -C /opt/bitnami --strip-components=2 --no-same-owner --wildcards '*/files' ; \
      rm -rf "${COMPONENT}".tar.gz{,.sha256} ; \
    done
RUN apt-get autoremove --purge -y curl && \
    apt-get update && apt-get upgrade -y && \
    apt-get clean && rm -rf /var/lib/apt/lists /var/cache/apt/archives
RUN chmod g+rwX /opt/bitnami
RUN find / -perm /6000 -type f -exec chmod a-s {} \; || true

COPY rootfs /
RUN chmod +x /opt/bitnami/scripts/redis-cluster/postunpack.sh && /opt/bitnami/scripts/redis-cluster/postunpack.sh
# RUN /opt/bitnami/scripts/redis-cluster/postunpack.sh
ENV APP_VERSION="7.2.4" \
    BITNAMI_APP_NAME="redis-cluster" \
    PATH="/opt/bitnami/common/bin:/opt/bitnami/redis/bin:$PATH"

# Set modules allowance
RUN echo "enable-module-command yes" >> /opt/bitnami/redis/etc/redis.conf

# ENV REDIS_EXTRA_FLAGS="--loadmodule /usr/lib/rejson.so --loadmodule /usr/lib/redisearch.so"
ENV REDIS_EXTRA_FLAGS="--loadmodule /opt/bitnami/redis/etc/rejson.so --loadmodule /opt/bitnami/redis/etc/redisearch.so --loadmodule /opt/bitnami/redis/etc/module-oss.so"
# ENV REDIS_EXTRA_FLAGS="--loadmodule /usr/lib/rejson.so --loadmodule /usr/lib/redisearch.so --loadmodule /usr/lib/redisai.so"
# ENV REDIS_EXTRA_FLAGS="--loadmodule /usr/lib/rejson.so"

# Set the execute permission for the entrypoint script
RUN chmod +x /opt/bitnami/scripts/redis-cluster/entrypoint.sh

# Set the execute permission for the setup script
RUN chmod +x /opt/bitnami/scripts/redis-cluster/setup.sh

# Set the execute permission for the run script
RUN chmod +x /opt/bitnami/scripts/redis-cluster/run.sh

EXPOSE 6379

USER 1001
ENTRYPOINT [ "/opt/bitnami/scripts/redis-cluster/entrypoint.sh" ]
CMD [ "/opt/bitnami/scripts/redis-cluster/run.sh" ]