docker-airflow
docker-airflow copied to clipboard
puckel
Cant run code as root
I am binding a directory from another container with root ownership from a python Operator i need to delete one file in that directory. I cant make it. If i change user owner or impersonate_user to root, i get this: 'No such file or directory: 'sudo': 'sudo'.
I have tried tons and tons of configurations but failed every single time, unless the directory's owner is airflow which is impossible for the other container to make.
@chris-aeviator my problem is that i want to run it from the beginning as root, to execute airflow as root not to just gain temporary access to the container as root.
I am facing similar issue i couldn't get root access and i need the root access to install some apps( like git) while running my dag files.
usually you should do that in a docker file while building the container. That means whenever you need more dependencies inside your airflow container(s), you'll edit your docker file and rebuild.
installing in the dockerfile is done with
RUN apt get install […] and then rebuild the docker container with docker build -t …
In the file Dockerfile
# FROM apache/airflow:2.2.1-python3.9
FROM puckel/docker-airflow:1.10.9
RUN apt-get update
RUN apt-get install -y git
when i try to build a new image shows the following error message.
`ERROR [2/3] RUN apt-get update 1.5s
[2/3] RUN apt-get update: #6 0.551 Reading package lists... #6 0.572 E: List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied) executor failed running [/bin/sh -c apt-get update]: exit code: 100 `
Try to rebuild the airflow Dockerfile (not FROM) and set
ADDITIONAL_DEV_APT_DEPS…airflow is a pain to setup for new use cases…
Maybe add USER root in the Dockerfile will help? Mine is like:
FROM apache/airflow:2.2.4
USER root
RUN apt-get update && \
apt-get upgrade -y && \
apt-get install -y git && \
apt-get install -y vim
USER airflow
COPY requirements.txt /requirements.txt
RUN pip3 install --user --upgrade pip
RUN pip3 install --no-cache-dir --user -r /requirements.txt
and i get git/vim installed in the end
In the file
Dockerfile
# FROM apache/airflow:2.2.1-python3.9FROM puckel/docker-airflow:1.10.9RUN apt-get updateRUN apt-get install -y gitwhen i try to build a new image shows the following error message.
`ERROR [2/3] RUN apt-get update 1.5s
[2/3] RUN apt-get update: #6 0.551 Reading package lists... #6 0.572 E: List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied) executor failed running [/bin/sh -c apt-get update]: exit code: 100 `
You should try with User Root because the image set Airflow for user default, and the update and upgrade run with root user only.
FROM puckel/docker-airflow:1.10.9
USER root
RUN apt-get update
RUN apt-get install -y git
USER airflow
Remember to change the user after installing. If you want to set up the airflow with sudo, just try:
USER root
RUN usermod -aG sudo airflow
RUN echo "airflow:123123" | chpasswd
and install subversion, sudo for container
without root is very hard to do many working. security is only important for Operation. No need for many use cases.
without root is very hard to do many working. security is only important for Operation. No need for many use cases.
Same here. I just a normal developer, want to start docker airflow with my project quickly. Why should I care about so many security issues even in docker container? I hope we can have an approach to work with root.
Well, for anyone is looking for a way to pip install package under root, you can simply do
python -m pip install {you lib} instead of pip install {you lib}.
