starrocks-kubernetes-operator icon indicating copy to clipboard operation
starrocks-kubernetes-operator copied to clipboard

Support for ReadOnlyRootFilesystem in Starrocks Chart

Open kapilsingh421 opened this issue 2 years ago • 2 comments

Describe the bug

Enabling ReadOnlyRootFilesystem should be supported.

To Reproduce

Not able to add ReadOnlyRootFilesystem:true

Expected behavior

Both the frontend and compute node are expected to function correctly.

The Persistent Volume (PV) paths are set as follows: for the frontend, /opt/starrocks/fe/meta and /opt/starrocks/fe/logs. The compute node uses a similar configuration with paths at /opt/starrocks/cn/storage and /opt/starrocks/cn/logs.

  • Chart Version V1.9.0

kapilsingh421 avatar Jan 05 '24 14:01 kapilsingh421

StarRocks(3.2) now does not support to run on ReadOnly file system. So we can not set pod.spec.containers.securityContext.readOnlyRootFilesystem to true.

In order to run StarRocks on read-only file system, we can do the following steps to work around this issue.

  1. Update Dockerfile. Copy the files from the original image to a temporary directory.
# specific starrocks fe-ubuntu image with the provided sha256 digest
FROM starrocks/fe-ubuntu:3.2-latest

# Set the environment variables for the source and destination directories
ENV STARROCKS_ROOT=/opt/starrocks
ENV STARROCKS_TEMP=/opt/starrocks_temp

RUN mkdir -p $STARROCKS_TEMP && cp -a $STARROCKS_ROOT/. $STARROCKS_TEMP/

# rename the original entrypoint script so it can be executed from the your own script
RUN mv $STARROCKS_TEMP/fe_entrypoint.sh $STARROCKS_ROOT/fe_entrypoint_orig.sh

# copy your own script to the container so it matches what the operator will create for the pod spec.cmd
COPY your-own-entrypoint.sh $STARROCKS_ROOT/fe_entrypoint.sh

# Note: this will make the size of the image larger than the original image
RUN chown -R starrocks:starrocks /opt/starrocks_temp/fe/
  1. Update entrypoint script. After we mount the persistent volume to the /opt/starrocks/fe directory, we can copy the files from the temporary directory to the persistent volume mount.
# The temporary location where we have our StarRocks files.
STARROCKS_TEMP="/opt/starrocks_temp"

# The persistent volume mount where StarRocks expects to find its files.
STARROCKS_ROOT="/opt/starrocks"

# Function to check if a directory is empty
is_dir_empty() {
    [ -z "$(ls -A "$1" 2>/dev/null)" ]
}

# Function to check and delete PID file
check_and_delete_pid() {
    local pid_dir="$1/bin"
    local pid_file=$(ls $pid_dir/*.pid 2>/dev/null)

    if [ ! -z "$pid_file" ]; then
        rm -f $pid_file
        echo "Deleted PID file: $pid_file"
    else
        echo "No PID file found in $pid_dir"
    fi
}

# Function to update /fe/lib and /fe/bin if the MD5 checksum of starrocks-fe.jar has changed.
update_fe_if_changed() {
    local src_jar="$STARROCKS_TEMP/fe/lib/starrocks-fe.jar"
    local dst_jar="$STARROCKS_ROOT/fe/lib/starrocks-fe.jar"

    if [ -f "$src_jar" ] && [ -f "$dst_jar" ]; then
        local src_md5=$(md5sum "$src_jar" | cut -d' ' -f1)
        local dst_md5=$(md5sum "$dst_jar" | cut -d' ' -f1)

        if [ "$src_md5" != "$dst_md5" ]; then
            echo "MD5 checksum does not match. Updating /fe/lib..."
            rm -rf "$STARROCKS_ROOT"/fe/lib
            cp -r "$STARROCKS_TEMP"/fe/lib "$STARROCKS_ROOT"/fe/
            cp -r "$STARROCKS_TEMP"/fe/bin "$STARROCKS_ROOT"/fe/
            echo "Updated /fe/lib successfully."
        else
            echo "MD5 checksum matches, no update needed for /fe/lib."
        fi
    fi
}

# Main logic for handling 'fe' directory copying and updating based on MD5 checksum comparison.
if [ ! -d "$STARROCKS_ROOT"/fe/bin ] || is_dir_empty "$STARROCKS_ROOT"/fe/bin; then
    echo "Bin directory is empty or does not exist in /fe, copying everything..."
    cp -r "$STARROCKS_TEMP"/fe/* "$STARROCKS_ROOT"/fe/
    echo "Files copied successfully to /fe."
else
    update_fe_if_changed
fi

# Check and delete PID files for 'fe'
check_and_delete_pid "$STARROCKS_ROOT/fe"

bash /opt/starrocks/fe_entrypoint_orig.sh "$FE_SERVICE_NAME"

yandongxiao avatar Jan 31 '24 07:01 yandongxiao

Thanks its a Good solution @yandongxiao

kapilsingh421 avatar Jan 31 '24 07:01 kapilsingh421