Zookeeper and Kafka volume chmod

[TaaviE]

Problem Statement

Currently install/update-docker-volume-permissions.sh contains docker run --rm -v "sentry-zookeeper:/sentry-zookeeper-data" -v "sentry-kafka:/sentry-kafka-data" -v "${COMPOSE_PROJECT_NAME}_sentry-zookeeper-log:/sentry-zookeeper-log-data" busybox chmod -R a+w /sentry-zookeeper-data /sentry-kafka-data /sentry-zookeeper-log-data

This can cause issues later on when non-external volumes are recreated or when coming from an older version of Sentry.

Solution Brainstorm

Would it be possible to turn this chmod into an one-off container that exits, some volume mount parameter or something more "automatic"?

[hubertdeng123]

This can cause issues later on when non-external volumes are recreated or when coming from an older version of Sentry.

Could you elaborate more on how this would lead to issues? If I'm understanding what you're asking for, I don't see how the container you're describing would do much different here, since it would still need the volume names in order to make these permission changes.

[TaaviE]

@hubertdeng123

Could you elaborate more on how this would lead to issues?

Without the chmod, Kafka and Zookeeper both fail to start.

The (minor) issue is that this modification to the volume is relatively hidden in the installation script. It's just that given how it's a prerequisite for those containers, it would be less error-prone if it were automatic so to say (somehow done by the main docker-compose.yml).

[hubertdeng123]

Without the chmod, Kafka and Zookeeper both fail to start.

Yes that is correct.

I did debate putting this in the docker-compose file. There would be two approaches there that came to mind for me, which I considered.

1. run kafka and zookeeper as root user
2. run the chmod in the docker compose file

The reason why I did not want to go with the first option is, I would rather not require running a docker container as root user for security reasons. The reason why I didn't go with the second option is placing the chmod in the install script is a one time thing. Placing it in the docker compose file will invoke the command every time services are brought up.

Note: This docker volume permissioning issue will not happen when docker volumes are recreated if you're on the newer version of zookeeper/kafka that we upgraded to. It will only occur after coming from older versions (at least when I tested this)

[TaaviE]

Note: This docker volume permissioning issue will not happen when docker volumes are recreated if you're on the newer version of zookeeper/kafka that we upgraded to. It will only occur after coming from older versions (at least when I tested this)

Oh! I guess this small stumble is okay then.

I just stumbled upon it while upgrading and it wasn't super straightforward back then how the issue occurred and how to fix it.