occlum
[BUG]Failed to run python code
When I install python environment, the shell script is below:
$script_dir/miniconda/bin/conda create --prefix $script_dir/python-occlum -y python=3.9.11 flask=2.2.2 flask-restful=0.3.9 jinja2=3.1.2 sqlalchemy=1.3.11 flask-sqlalchemy=2.5.1 pymysql=0.9.3 numpy=1.21.5 scipy=1.7.3 scikit-learn=1.0 pandas=1.3 rsa=4.7.2
However, the errors shows Conflicts.
solving environment: failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): done Solving environment: | Found conflicts! Looking for incompatible packages. This can take several minutes. Press CTRL-C to abort. failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Output in format: Requested package -> Available versions
Package markupsafe conflicts for: flask=2.2.2 -> jinja2[version='>=3.0'] -> markupsafe[version='>=2.0|>=2.1.1'] jinja2=3.1.2 -> markupsafe[version='>=2.0']
Package bzip2 conflicts for: flask=2.2.2 -> python[version='>=3.10,<3.11.0a0'] -> bzip2[version='>=1.0.8,<2.0a0'] scipy=1.7.3 -> python[version='>=3.10,<3.11.0a0'] -> bzip2[version='>=1.0.8,<2.0a0'] flask-sqlalchemy=2.5.1 -> python -> bzip2[version='>=1.0.8,<2.0a0'] rsa=4.7.2 -> python -> bzip2[version='>=1.0.8,<2.0a0'] numpy=1.21.5 -> python[version='>=3.10,<3.11.0a0'] -> bzip2[version='>=1.0.8,<2.0a0'] scikit-learn=1.0 -> python[version='>=3.10,<3.11.0a0'] -> bzip2[version='>=1.0.8,<2.0a0'] jinja2=3.1.2 -> python[version='>=3.10,<3.11.0a0'] -> bzip2[version='>=1.0.8,<2.0a0'] flask-restful=0.3.9 -> python[version='>=3.10,<3.11.0a0'] -> bzip2[version='>=1.0.8,<2.0a0']
Package _libgcc_mutex conflicts for: scipy=1.7.3 -> libgcc-ng[version='>=11.2.0'] -> _libgcc_mutex[version='|0.1',build=main] sqlalchemy=1.3.11 -> libgcc-ng[version='>=7.3.0'] -> _libgcc_mutex[version='|0.1',build=main] python=3.9.11 -> libgcc-ng[version='>=7.5.0'] -> _libgcc_mutex[version='|0.1',build=main] pandas=1.3 -> libgcc-ng[version='>=7.5.0'] -> _libgcc_mutex[version='|0.1',build=main] scikit-learn=1.0 -> _openmp_mutex -> _libgcc_mutex[version='|0.1',build=main] numpy=1.21.5 -> libgcc-ng[version='>=11.2.0'] -> _libgcc_mutex[version='|0.1',build=main]
Package tzdata conflicts for: flask-restful=0.3.9 -> python[version='>=3.9,<3.10.0a0'] -> tzdata jinja2=3.1.2 -> python[version='>=3.9,<3.10.0a0'] -> tzdata pandas=1.3 -> python[version='>=3.9,<3.10.0a0'] -> tzdata scikit-learn=1.0 -> python[version='>=3.9,<3.10.0a0'] -> tzdata rsa=4.7.2 -> python -> tzdata scipy=1.7.3 -> python[version='>=3.9,<3.10.0a0'] -> tzdata numpy=1.21.5 -> python[version='>=3.9,<3.10.0a0'] -> tzdata python=3.9.11 -> tzdata flask-sqlalchemy=2.5.1 -> python -> tzdata flask=2.2.2 -> python[version='>=3.10,<3.11.0a0'] -> tzdata
Package python conflicts for: pandas=1.3 -> python[version='>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0|>=3.7.1,<3.8.0a0'] rsa=4.7.2 -> python scikit-learn=1.0 -> python[version='>=3.10,<3.11.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0'] pymysql=0.9.3 -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0'] rsa=4.7.2 -> pyasn1[version='>=0.1.3'] -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0'] sqlalchemy=1.3.11 -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.8,<3.9.0a0|>=3.7,<3.8.0a0'] pymysql=0.9.3 -> cryptography -> python[version='>=3.10,<3.11.0a0|>=3.9,<3.10.0a0|>=3.8,<3.9.0a0|>=3.5,<3.6.0a0'] flask=2.2.2 -> click[version='>=8.0'] -> python[version='>=3.6|>=3.6,<3.7.0a0'] flask=2.2.2 -> python[version='>=3.10,<3.11.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0'] flask-restful=0.3.9 -> aniso8601[version='>=0.82'] -> python[version='>=2.7,<2.8.0a0|>=3.5|>=3.6|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0'] jinja2=3.1.2 -> markupsafe[version='>=2.0'] -> python[version='>=3.6,<3.7.0a0'] jinja2=3.1.2 -> python[version='>=3.10,<3.11.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0|>=3.7,<3.8.0a0'] flask-sqlalchemy=2.5.1 -> flask[version='>=1.0.4'] -> python[version='>=2.7,<2.8.0a0|>=3.10,<3.11.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0|>=3.6|>=3.5|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0'] flask-restful=0.3.9 -> python[version='>=3.10,<3.11.0a0|>=3.9,<3.10.0a0|>=3.8,<3.9.0a0|>=3.7,<3.8.0a0'] python=3.9.11 scipy=1.7.3 -> python[version='>=3.10,<3.11.0a0|>=3.7,<3.8.0a0|>=3.9,<3.10.0a0|>=3.8,<3.9.0a0'] scikit-learn=1.0 -> joblib[version='>=0.11'] -> python[version='>=2.7,<2.8.0a0|>=3.6|>=3.7|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0|>=3.5'] numpy=1.21.5 -> python[version='>=3.10,<3.11.0a0|>=3.9,<3.10.0a0|>=3.8,<3.9.0a0|>=3.7,<3.8.0a0'] pandas=1.3 -> bottleneck[version='>=1.2.1'] -> python[version='>=2.7,<2.8.0a0|>=3.10,<3.11.0a0|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0'] numpy=1.21.5 -> mkl-service[version='>=2.3.0,<3.0a0'] -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0'] scipy=1.7.3 -> mkl-service[version='>=2.3.0,<3.0a0'] -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0'] flask-sqlalchemy=2.5.1 -> python
Package libgfortran4 conflicts for: scikit-learn=1.0 -> scipy[version='>=1.1.0'] -> libgfortran4[version='>=7.5.0'] scipy=1.7.3 -> libgfortran-ng -> libgfortran4=7.5.0 scipy=1.7.3 -> libgfortran4[version='>=7.5.0'] numpy=1.21.5 -> libopenblas[version='>=0.3.3,<1.0a0'] -> libgfortran4[version='>=7.5.0']
Package ca-certificates conflicts for: python=3.9.11 -> openssl[version='>=1.1.1n,<1.1.2a'] -> ca-certificates pymysql=0.9.3 -> python[version='>=2.7,<2.8.0a0'] -> ca-certificates rsa=4.7.2 -> python -> ca-certificates flask-sqlalchemy=2.5.1 -> python -> ca-certificates sqlalchemy=1.3.11 -> python[version='>=2.7,<2.8.0a0'] -> ca-certificates
Package sqlalchemy conflicts for: flask-sqlalchemy=2.5.1 -> sqlalchemy[version='>=1.2'] sqlalchemy=1.3.11
Package six conflicts for: pymysql=0.9.3 -> cryptography -> six[version='>=1.4.1'] flask-restful=0.3.9 -> six[version='>=1.3.0'] numpy=1.21.5 -> mkl-service[version='>=2.3.0,<3.0a0'] -> six pandas=1.3 -> python-dateutil[version='>=2.7.3'] -> six[version='>=1.5'] scipy=1.7.3 -> mkl-service[version='>=2.3.0,<3.0a0'] -> six
Package numpy conflicts for: scipy=1.7.3 -> numpy[version='>=1.16,<1.23|>=1.21,<1.23|>=1.16.6,<1.23.0|>=1.21.2,<1.23.0'] numpy=1.21.5 pandas=1.3 -> bottleneck[version='>=1.2.1'] -> numpy[version='>=1.11.3,<2.0a0|>=1.14.6,<2.0a0|>=1.16.6,<2.0a0|>=1.21.5,<2.0a0|>=1.21.2,<2.0a0|>=1.9.3,<2.0a0|>=1.16.4,<2.0a0'] scikit-learn=1.0 -> numpy[version='>=1.16.6,<2.0a0|>=1.21.2,<2.0a0'] pandas=1.3 -> numpy[version='>=1.19.2,<2.0a0|>=1.20.3,<2.0a0'] scikit-learn=1.0 -> scipy[version='>=1.1.0'] -> numpy[version='>=1.11.3,<2.0a0|>=1.14.6,<2.0a0|>=1.16,<1.23|>=1.19,<1.25.0|>=1.19,<1.26.0|>=1.19,<1.27.0|>=1.21,<1.27.0|>=1.21,<1.26.0|>=1.21,<1.25.0|>=1.21,<1.23|>=1.16.6,<1.23.0|>=1.21.2,<1.23.0|>=1.15.1,<2.0a0|>=1.9.3,<2.0a0']
Package jinja2 conflicts for: flask-sqlalchemy=2.5.1 -> flask[version='>=1.0.4'] -> jinja2[version='>=2.10,<3.0|>=2.10.1,<3.0|>=3.0'] flask=2.2.2 -> jinja2[version='>=3.0'] jinja2=3.1.2 flask-restful=0.3.9 -> flask[version='>=0.8'] -> jinja2[version='>=2.10,<3.0|>=2.10.1,<3.0|>=3.0|>=2.4']
Package setuptools conflicts for: scikit-learn=1.0 -> joblib[version='>=0.11'] -> setuptools pandas=1.3 -> numexpr[version='>=2.7.0'] -> setuptools flask=2.2.2 -> jinja2[version='>=3.0'] -> setuptools python=3.9.11 -> pip -> setuptools
Package flask conflicts for: flask=2.2.2 flask-restful=0.3.9 -> flask[version='>=0.8'] flask-sqlalchemy=2.5.1 -> flask[version='>=1.0.4']
Package scipy conflicts for: scipy=1.7.3 scikit-learn=1.0 -> scipy[version='>=1.1.0']
Package pytz conflicts for: pandas=1.3 -> pytz[version='>=2017.3'] flask-restful=0.3.9 -> pytzThe following specifications were found to be incompatible with your system:
- feature:/linux-64::__glibc==2.31=0
- feature:|@/linux-64::__glibc==2.31=0
- numpy=1.21.5 -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
- pandas=1.3 -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
- scikit-learn=1.0 -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
- scipy=1.7.3 -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
- sqlalchemy=1.3.11 -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
Your installed version is: 2.31
What should I do? Does it support any other method to set up python environment in Occlum beyond miniconda? I truely found package in offcial website but the console always said: not found package. It's ridiculous.
@Klein920116 The python package conflicts issue is beyond Occlum. Generally, Occlum suggests to use miniconda for python applications because miniconda can pack all the files required in one directory which can ease the Occlum image FS creation. But you can try any other method to set up python environment (Occlum has no recommendation). BTW, for the conflicts perhaps you can find answers on google, such as https://stackoverflow.com/questions/62288835/how-to-interpret-conda-package-conflicts
@Klein920116 The python package conflicts issue is beyond Occlum. Generally, Occlum suggests to use miniconda for python applications because miniconda can pack all the files required in one directory which can ease the Occlum image FS creation. But you can try any other method to set up python environment (Occlum has no recommendation). BTW, for the conflicts perhaps you can find answers on google, such as https://stackoverflow.com/questions/62288835/how-to-interpret-conda-package-conflicts
Thanks, the problem has been solved. @qzheng527
Does anyone encounter this error? When I run the python code, the error shows memory allocation failed. OSError: [Errno 5] Input/output error memory allocation of 4096 bytes failed /opt/occlum/build/bin/occlum: line 354: 9987 Illegal instruction (core dumped) RUST_BACKTRACE=1 "$instance_dir/build/bin/occlum-run" "$@"
@Klein920116 Please try increase the "resource_limits.kernel_space_heap_size" in Occlum.json then do "occlum build" again. Details you can refer to doc.
@Klein920116 Please try increase the "resource_limits.kernel_space_heap_size" in Occlum.json then do "occlum build" again. Details you can refer to doc.
@qzheng527 Cool, the problem has been solved.