scenic
scenic copied to clipboard
Dockerfile
Dear all,
I hope you are doing well. Can you provide a docker file to run scenic? It looks like there are quite a lot of deps (even with tensorflow 🤢) for jax-only models. It would help the community a lot, improving usability and sharability.
Thanks,
Francesco
Hi Francesco,
A docker file sounds like a great idea! Unfortunately, I have little experience with Docker. Would you be willing to draft a docker file based on the quickstart instructions? Once we have a starting point, I'm happy to test and debug it to make it work.
Thanks, Matthias
Ah I saw that you provided this in #533, I'll test this.
@mjlm happy to help! Let me try to find sometime this we :)
@mjlm happy to help! Let me try to find sometime this we :)
i can provide my Dockerfile.
FROM nvidia/cuda:11.7.1-cudnn8-devel-ubuntu20.04
ENV TZ Asia/Seoul
ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A4B469963BF863CC
RUN rm /etc/apt/sources.list.d/*
RUN apt-get update && apt-get -y install \
python3 python3-pip python3-dev \
jupyter-notebook \
git ssh vim
RUN apt install -y software-properties-common
RUN add-apt-repository ppa:deadsnakes/ppa
RUN apt-get install -y python3.9
RUN ln -sf /usr/share/zoneinfo/Asia/Seoul /etc/localtime
RUN ln -s /usr/bin/python3 /usr/bin/python
RUN echo "root:root" | chpasswd
RUN sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes #prohibit-password/' /etc/ssh/sshd_config
RUN pip3 install --upgrade pip
RUN pip3 install setuptools
RUN apt-get update && apt-get install -y locales
RUN locale-gen ko_KR.UTF-8
ENV LC_ALL ko_KR.UTF-8
WORKDIR /workspace
ADD . .
RUN jupyter notebook --generate-config
RUN echo "c.NotebookApp.password = 'key'" >> /root/.jupyter/jupyter_notebook_config.py
RUN echo "c.NotebookApp.ip = '0.0.0.0'" >> /root/.jupyter/jupyter_notebook_config.py
RUN echo "c.NotebookApp.allow_root = True" >> /root/.jupyter/jupyter_notebook_config.py
RUN echo "c.NotebookApp.open_browser = False" >> /root/.jupyter/jupyter_notebook_config.py
RUN echo "c.NotebookApp.port = 8888" >> /root/.jupyter/jupyter_notebook_config.py
ENV PYTHONPATH $PYTHONPATH:/workspace
RUN chmod -R a+w /workspace
RUN alias python=python3.9
For Python 3.9 will require you to tweak it a little bit more from this script.(When I use docker, I usually pay more attention to the CUDA version.) Good luck~
@BigJoon neat, thanks a lot :pray:
I have created a Dockerfile for the project with a focus on ViVit pipeline. The key challenge was to match the right versions of cudann (11), tensorrt (7). The proper cudann, tensorrt version and respective docker container version can be determined from the table available at: https://docs.nvidia.com/deeplearning/tensorrt/container-release-notes/rel-22-12.html#rel-22-12. Hopefully it helps the users of the project
FROM nvcr.io/nvidia/tensorrt:20.11-py3
# From the version table https://docs.nvidia.com/deeplearning/tensorrt/container-release-notes/rel-22-12.html#rel-22-12
# Select appropriate container version with proper cuda version and tensorrt framework version
# For me it was 20.11
RUN apt update -y
# The following libraries are need to compile and install python3.9 manually
RUN apt install build-essential zlib1g-dev libncurses5-dev \
libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev \
libsqlite3-dev wget libbz2-dev liblzma-dev -y
# Steps to install python:3.9
WORKDIR /app
RUN wget https://www.python.org/ftp/python/3.9.7/Python-3.9.7.tgz
RUN tar -xvf Python-3.9.7.tgz
WORKDIR /app/Python-3.9.7/
RUN ./configure --enable-optimizations
RUN make
RUN make altinstall
#RUN configure && make && make install
# Since python 3.9 is required for the project, replacing any older python3
RUN cp /usr/local/bin/python3.9 /usr/bin/python3 && \
cp /usr/local/bin/python3.9 /usr/bin/python
# Installing pip for python3.9
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
RUN python3.9 get-pip.py
# Here we are setting up the project, with a focus on ViViT
WORKDIR /workspace
RUN cd /workspace && git clone https://github.com/google-research/scenic.git &&\
cd scenic &&\
sed -i '/ "tf-models-official",/d' ./setup.py &&\
python3 -m pip install -q . &&\
python3 -m pip install -r scenic/projects/baselines/clip/requirements.txt &&\
python3 -m pip install scikit-image
WORKDIR /workspace/scenic
RUN pip install -r scenic/projects/vivit/requirements.txt
# manually install some libraries specifically needed by ViViT
RUN python3 -m pip install --upgrade "jax[cuda]==0.3.1" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
RUN pip install flax==0.4.0 sklearn scikit-learn tf-models-official==2.8.0 tensorflow-gpu==2.8.0
@dipodaimary Hi Thank you very much for your Dockerfile. I tried it. but I have warnings with the NVIDIA Driver. I have attached the warning message for your perusal. Can you help me solve this problem?
===================== == NVIDIA TensorRT ==
NVIDIA Release 20.11 (build 17147175)
NVIDIA TensorRT 7.2.1 (c) 2016-2020, NVIDIA CORPORATION. All rights reserved. Container image (c) 2020, NVIDIA CORPORATION. All rights reserved.
https://developer.nvidia.com/tensorrt
To install Python sample dependencies, run /opt/tensorrt/python/python_setup.sh
To install the open-source samples corresponding to this TensorRT release version run /opt/tensorrt/install_opensource.sh.
To build the open source parsers, plugins, and samples for current top-of-tree on master or a different branch, run /opt/tensorrt/install_opensource.sh -b
WARNING: The NVIDIA Driver was not detected. GPU functionality will not be available. Use 'nvidia-docker run' to start this container; see https://github.com/NVIDIA/nvidia-docker/wiki/nvidia-docker .