schemachange
schemachange copied to clipboard
Snowflake-Labs
Getting error while running schemachange docker on gitlab cicd
Describe the bug While running the docker command in gitlab cicd to run schemachange, we are getting error. Please find the gitlab stage details below
deploy-to-eu-staging-snowflake: stage: snowflake-staging when: manual image: docker:latest services: - docker:dind environment: name: staging before_script: - cat contents/A__insert_data.sql script: - > docker run --rm --name schemachange-script -v "$PWD":/usr/src/schemachange -w /usr/src/schemachange -e SNOWFLAKE_USER -e SNOWFLAKE_ROLE -e SNOWFLAKE_PRIVATE_KEY_PATH -e SNOWFLAKE_PRIVATE_KEY -e SNOWFLAKE_AUTHENTICATOR python:3 /bin/bash -c "export OPENBLAS_NUM_THREADS=1 && pip install schemachange --upgrade --progress-bar off --debug && schemachange -f contents/ -a SNOWFLAKE_URL -u $SNOWFLAKE_USER -r $SNOWFLAKE_ROLE -w TEST_WH -d TEST_DB -c TEST_DB.SCHEMACHANGE.CHANGE_HISTORY"
Steps to reproduce the behavior:
- Go to 'gitlab pi'
- Click on 'Run pipeline'
- Scroll down to 'log'
- See error
Expected behavior A clear and concise description of what you expected to happen.
Screenshots
Schemachange (please complete the following information):
- Version (e.g. 3.5.3): 3.6.0-py3
Additional context Please find the details logs in the gitlab cicd
$ docker run --rm --name schemachange-script -v "$PWD":/usr/src/schemachange -w /usr/src/schemachange -e SNOWFLAKE_USER -e SNOWFLAKE_ROLE -e SNOWFLAKE_PRIVATE_KEY_PATH -e SNOWFLAKE_PRIVATE_KEY -e SNOWFLAKE_AUTHENTICATOR python:3 /bin/bash -c "export OPENBLAS_NUM_THREADS=1 && pip install schemachange --upgrade --progress-bar off --debug && schemachange -f contents/ -a SNOWFLAKE_URL -u $SNOWFLAKE_USER -r $SNOWFLAKE_ROLE -w TEST_WH -d TEST_DB -c TEST_DB.SCHEMACHANGE.CHANGE_HISTORY" Unable to find image 'python:3' locally 3: Pulling from library/python 8457fd5474e7: Pulling fs layer 13baa2029dde: Pulling fs layer 325c5bf4c2f2: Pulling fs layer 7e18a660069f: Pulling fs layer 98a59f0ffede: Pulling fs layer 72c7f17f2221: Pulling fs layer 2f40b346325a: Pulling fs layer f3f08e04e337: Pulling fs layer
Digest: sha256:7b8d65a924f596eb65306214f559253c468336bcae09fd575429774563460caf Status: Downloaded newer image for python:3 Collecting schemachange Obtaining dependency information for schemachange from https://files.pythonhosted.org/packages/91/36/828c2caac9933af76893e3f73fe5e63f0b100e49bf1aa52c48160ceb87fd/schemachange-3.6.0-py3-none-any.whl.metadata Downloading schemachange-3.6.0-py3-none-any.whl.metadata (33 kB) Collecting jinja2~=3.0 (from schemachange) Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB) Collecting pandas~=1.3 (from schemachange) Downloading pandas-1.5.3.tar.gz (5.2 MB) Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: still running... Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Collecting pyyaml~=6.0 (from schemachange) Obtaining dependency information for pyyaml~=6.0 from https://files.pythonhosted.org/packages/b4/33/720548182ffa8344418126017aa1d4ab4aeec9a2275f04ce3f3573d8ace8/PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata Downloading PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB) Collecting snowflake-connector-python<4.0,>=2.8 (from schemachange) Downloading snowflake-connector-python-3.4.0.tar.gz (719 kB) Installing build dependencies: started Installing build dependencies: finished with status 'error' error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully. │ exit code: 1 ╰─> [331 lines of output] Collecting setuptools>=40.6.0 Obtaining dependency information for setuptools>=40.6.0 from https://files.pythonhosted.org/packages/bb/26/7945080113158354380a12ce26873dd6c1ebd88d47f5bc24e2c5bb38c16a/setuptools-68.2.2-py3-none-any.whl.metadata Using cached setuptools-68.2.2-py3-none-any.whl.metadata (6.3 kB) Collecting wheel Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/fa/7f/4c07234086edbce4a0a446209dc0cb08a19bb206a3ea53b2f56a403f983b/wheel-0.41.3-py3-none-any.whl.metadata Using cached wheel-0.41.3-py3-none-any.whl.metadata (2.2 kB) Collecting cython Obtaining dependency information for cython from https://files.pythonhosted.org/packages/0a/f5/a629b0decba74c6cd1f9e830d1e55b89b6be15507d26c7cc14bc87275b28/Cython-3.0.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata Downloading Cython-3.0.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.2 kB) Collecting pyarrow<10.1.0,>=10.0.1 Downloading pyarrow-10.0.1.tar.gz (994 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 994.1/994.1 kB 15.4 MB/s eta 0:00:00 Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Collecting numpy>=1.16.6 (from pyarrow<10.1.0,>=10.0.1) Obtaining dependency information for numpy>=1.16.6 from https://files.pythonhosted.org/packages/07/c0/ccbb2a4c75b283d6100400a907087bfa4d89cee9df73fa6af85268115d81/numpy-1.26.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata Using cached numpy-1.26.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB) Using cached setuptools-68.2.2-py3-none-any.whl (807 kB) Using cached wheel-0.41.3-py3-none-any.whl (65 kB) Using cached Cython-3.0.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.5 MB) Using cached numpy-1.26.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.9 MB) Building wheels for collected packages: pyarrow Building wheel for pyarrow (pyproject.toml): started Building wheel for pyarrow (pyproject.toml): finished with status 'error' error: subprocess-exited-with-error
× Building wheel for pyarrow (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [289 lines of output]
<string>:36: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
WARNING setuptools_scm.pyproject_reading toml section missing 'pyproject.toml does not contain a tool.setuptools_scm section'
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-312
creating build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/filesystem.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_generated_version.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/plasma.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/csv.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/types.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/__init__.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/cuda.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/benchmark.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/flight.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/cffi.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/conftest.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/substrait.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/json.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/ipc.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/pandas_compat.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/hdfs.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/feather.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/jvm.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/serialization.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/util.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/orc.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/compute.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/fs.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/dataset.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_compute_docstrings.py -> build/lib.linux-x86_64-cpython-312/pyarrow
creating build/lib.linux-x86_64-cpython-312/pyarrow/parquet
copying pyarrow/parquet/core.py -> build/lib.linux-x86_64-cpython-312/pyarrow/parquet
copying pyarrow/parquet/__init__.py -> build/lib.linux-x86_64-cpython-312/pyarrow/parquet
copying pyarrow/parquet/encryption.py -> build/lib.linux-x86_64-cpython-312/pyarrow/parquet
creating build/lib.linux-x86_64-cpython-312/pyarrow/vendored
copying pyarrow/vendored/__init__.py -> build/lib.linux-x86_64-cpython-312/pyarrow/vendored
copying pyarrow/vendored/docscrape.py -> build/lib.linux-x86_64-cpython-312/pyarrow/vendored
copying pyarrow/vendored/version.py -> build/lib.linux-x86_64-cpython-312/pyarrow/vendored
creating build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_extension_type.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_cuda.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_flight.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_schema.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_convert_builtin.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_types.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_csv.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_exec_plan.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_json.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_serialization_deprecated.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_table.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/strategies.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_cpp_internals.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_sparse_tensor.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_hdfs.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_deprecations.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_serialization.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/__init__.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_tensor.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_substrait.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/read_record_batch.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_cython.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/deserialize_buffer.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/conftest.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_orc.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_plasma_tf_op.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/pandas_threaded_import.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_dataset.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_io.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_filesystem.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/arrow_7980.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_scalars.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_misc.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_builder.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_plasma.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_cffi.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_adhoc_memory_leak.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_jvm.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/arrow_16597.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_ipc.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_util.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_cuda_numba_interop.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_array.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/util.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_memory.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_gandiva.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_strategies.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/pandas_examples.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_udf.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_compute.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_fs.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_feather.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_gdb.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_pandas.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
creating build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_datetime.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_data_types.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_parquet_writer.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/common.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/__init__.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_parquet_file.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/encryption.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/conftest.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_dataset.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_metadata.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_basic.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_compliant_nested_type.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_encryption.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_pandas.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
running egg_info
writing pyarrow.egg-info/PKG-INFO
writing dependency_links to pyarrow.egg-info/dependency_links.txt
writing entry points to pyarrow.egg-info/entry_points.txt
writing requirements to pyarrow.egg-info/requires.txt
writing top-level names to pyarrow.egg-info/top_level.txt
ERROR setuptools_scm._file_finders.git listing git files failed - pretending there aren't any
reading manifest file 'pyarrow.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '../LICENSE.txt'
warning: no files found matching '../NOTICE.txt'
warning: no previously-included files matching '*.so' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*~' found anywhere in distribution
warning: no previously-included files matching '#*' found anywhere in distribution
warning: no previously-included files matching '.git*' found anywhere in distribution
warning: no previously-included files matching '.DS_Store' found anywhere in distribution
no previously-included directories found matching '.asv'
writing manifest file 'pyarrow.egg-info/SOURCES.txt'
copying pyarrow/__init__.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_compute.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_compute.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_csv.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_csv.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_cuda.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_cuda.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_dataset.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_dataset.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_dataset_orc.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_dataset_parquet.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_exec_plan.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_feather.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_flight.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_fs.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_fs.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_gcsfs.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_hdfs.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_hdfsio.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_json.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_orc.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_orc.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_parquet.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_parquet.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_parquet_encryption.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_parquet_encryption.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_plasma.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_pyarrow_cpp_tests.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_pyarrow_cpp_tests.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_s3fs.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_substrait.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/array.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/benchmark.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/builder.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/compat.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/config.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/error.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/gandiva.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/io.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/ipc.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/lib.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/lib.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/memory.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/pandas-shim.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/public-api.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/scalar.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/serialization.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/table.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/tensor.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/types.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
creating build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow_dataset.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libplasma.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow_python.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/__init__.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/common.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow_fs.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow_substrait.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow_dataset_parquet.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow_flight.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow_cuda.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow_feather.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libgandiva.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
creating build/lib.linux-x86_64-cpython-312/pyarrow/src
copying pyarrow/src/ArrowPythonConfig.cmake.in -> build/lib.linux-x86_64-cpython-312/pyarrow/src
copying pyarrow/src/ArrowPythonFlightConfig.cmake.in -> build/lib.linux-x86_64-cpython-312/pyarrow/src
copying pyarrow/src/CMakeLists.txt -> build/lib.linux-x86_64-cpython-312/pyarrow/src
copying pyarrow/src/arrow-python-flight.pc.in -> build/lib.linux-x86_64-cpython-312/pyarrow/src
copying pyarrow/src/arrow-python.pc.in -> build/lib.linux-x86_64-cpython-312/pyarrow/src
creating build/lib.linux-x86_64-cpython-312/pyarrow/tensorflow
copying pyarrow/tensorflow/plasma_op.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/tensorflow
copying pyarrow/tests/bound_function_visit_strings.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/pyarrow_cython_example.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
creating build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow
creating build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/CMakeLists.txt -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/api.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/arrow_to_pandas.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/arrow_to_pandas.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/arrow_to_python_internal.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/benchmark.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/benchmark.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/common.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/common.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/csv.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/csv.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/datetime.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/datetime.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/decimal.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/decimal.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/deserialize.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/deserialize.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/extension_type.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/extension_type.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/filesystem.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/filesystem.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/flight.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/flight.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/gdb.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/gdb.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/helpers.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/helpers.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/inference.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/inference.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/init.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/init.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/io.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/io.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/ipc.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/ipc.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/iterators.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/numpy_convert.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/numpy_convert.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/numpy_internal.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/numpy_interop.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/numpy_to_arrow.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/numpy_to_arrow.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/parquet_encryption.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/parquet_encryption.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/pch.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/platform.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/pyarrow.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/pyarrow.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/pyarrow_api.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/pyarrow_lib.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/python_test.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/python_test.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/python_to_arrow.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/python_to_arrow.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/serialize.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/serialize.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/type_traits.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/udf.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/udf.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/visibility.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
creating build/lib.linux-x86_64-cpython-312/pyarrow/tests/data
creating build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/README.md -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/TestOrcFile.emptyFile.jsn.gz -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/TestOrcFile.emptyFile.orc -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/TestOrcFile.test1.jsn.gz -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/TestOrcFile.test1.orc -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/TestOrcFile.testDate1900.jsn.gz -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/TestOrcFile.testDate1900.orc -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/decimal.jsn.gz -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/decimal.orc -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
creating build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/parquet
copying pyarrow/tests/data/parquet/v0.7.1.all-named-index.parquet -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/parquet
copying pyarrow/tests/data/parquet/v0.7.1.column-metadata-handling.parquet -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/parquet
copying pyarrow/tests/data/parquet/v0.7.1.parquet -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/parquet
copying pyarrow/tests/data/parquet/v0.7.1.some-named-index.parquet -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/parquet
creating build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/feather
copying pyarrow/tests/data/feather/v0.17.0.version.2-compression.lz4.feather -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/feather
running build_ext
creating /tmp/pip-install-2fn6embz/pyarrow_ad0ba6d0cd4345898a1cedc4dd36a4f6/build/cpp
-- Running CMake for PyArrow C++
cmake -DARROW_BUILD_DIR=build -DCMAKE_BUILD_TYPE=release -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_PREFIX=/tmp/pip-install-2fn6embz/pyarrow_ad0ba6d0cd4345898a1cedc4dd36a4f6/build/dist -DPYTHON_EXECUTABLE=/usr/local/bin/python -DPython3_EXECUTABLE=/usr/local/bin/python -DPYARROW_CXXFLAGS= -DPYARROW_WITH_DATASET=off -DPYARROW_WITH_PARQUET_ENCRYPTION=off -DPYARROW_WITH_HDFS=off -DPYARROW_WITH_FLIGHT=off /tmp/pip-install-2fn6embz/pyarrow_ad0ba6d0cd4345898a1cedc4dd36a4f6/pyarrow/src
error: command 'cmake' failed: No such file or directory
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pyarrow
Failed to build pyarrow
ERROR: Could not build wheels for pyarrow, which is required to install pyproject.toml-based projects
[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: pip install --upgrade pip
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip. [notice] A new release of pip is available: 23.2.1 -> 23.3.1 [notice] To update, run: pip install --upgrade pip ╭───────────────────── Traceback (most recent call last) ─────────────────────────────────╮ │ │ │ args = [ │ │ │ │ │ '/usr/local/bin/python', │ │ │ │ │ │ │ │ │ '/usr/local/lib/python3.12/site-packages/pip/__pip-run… │ │ │ │ │ 'install', │ │ │ │ │ '--ignore-installed', │ │ │ │ │ '--no-user', │ │ │ │ │ '--prefix', │ │ │ │ │ '/tmp/pip-build-env-21a4wy6w/overlay', │ │ │ │ │ '--no-warn-script-location', │ │ │ │ │ '--no-binary', │ │ │ │ │ ':none:', │ │ │ │ │ ... +9 │ │ │ │ ] │ │ │ │ extra_environ = { │ │ │ │ │ '_PIP_STANDALONE_CERT': │ │ │ │ '/usr/local/lib/python3.12/site-packages/pip/_vendor/c… │ │ │ │ } │ │ │ │ finder = <pip._internal.index.package_finder.PackageFinder │ │ │ │ object at 0x7f3aeb1fc0b0> │ │ │ │ format_control = 'only_binary' │ │ │ │ formats = set() │ │ │ │ index_urls = ['https://pypi.org/simple'] │ │ │ │ kind = 'build dependencies' │ │ │ │ pip_runnable = '/usr/local/lib/python3.12/site-packages/pip/__pip-run… │ │ │ │ prefix = <pip._internal.build_env._Prefix object at │ │ │ │ 0x7f3ae9b0b380> │ │ │ │ requirements = [ │ │ │ │ │ 'setuptools>=40.6.0', │ │ │ │ │ 'wheel', │ │ │ │ │ 'cython', │ │ │ │ │ 'pyarrow>=10.0.1,<10.1.0' │ │ │ │ ] │ │ │ │ spinner = <pip._internal.cli.spinners.NonInteractiveSpinner │ │ │ │ object at 0x7f3ae9b0b590> │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /usr/local/lib/python3.12/site-packages/pip/_internal/utils/subprocess.py:22 │ │ 4 in call_subprocess │ │ │ │ 221 │ │ │ │ │ extra={"markup": True}, │ │ 222 │ │ │ │ ) │ │ 223 │ │ │ │ │ ❱ 224 │ │ │ raise error │ │ 225 │ │ elif on_returncode == "warn": │ │ 226 │ │ │ subprocess_logger.warning( │ │ 227 │ │ │ │ 'Command "%s" had error code %s in %s', │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ all_output = [ │ │ │ │ │ 'Collecting setuptools>=40.6.0\n', │ │ │ │ │ ' Obtaining dependency information for │ │ │ │ setuptools>=40.6.0 from https://files.pyt'+134, │ │ │ │ │ ' Using cached │ │ │ │ setuptools-68.2.2-py3-none-any.whl.metadata (6.3 │ │ │ │ kB)\n', │ │ │ │ │ 'Collecting wheel\n', │ │ │ │ │ ' Obtaining dependency information for wheel │ │ │ │ from https://files.pythonhosted.org'+116, │ │ │ │ │ ' Using cached │ │ │ │ wheel-0.41.3-py3-none-any.whl.metadata (2.2 │ │ │ │ kB)\n', │ │ │ │ │ 'Collecting cython\n', │ │ │ │ │ ' Obtaining dependency information for │ │ │ │ cython from https://files.pythonhosted.or'+159, │ │ │ │ │ ' Downloading │ │ │ │ Cython-3.0.5-cp312-cp312-manylinux_2_17_x86_64.m… │ │ │ │ │ 'Collecting pyarrow<10.1.0,>=10.0.1\n', │ │ │ │ │ ... +321 │ │ │ │ ] │ │ │ │ cmd = [ │ │ │ │ │ '/usr/local/bin/python', │ │ │ │ │ │ │ │ │ '/usr/local/lib/python3.12/site-packages/pip/__p… │ │ │ │ │ 'install', │ │ │ │ │ '--ignore-installed', │ │ │ │ │ '--no-user', │ │ │ │ │ '--prefix', │ │ │ │ │ '/tmp/pip-build-env-21a4wy6w/overlay', │ │ │ │ │ '--no-warn-script-location', │ │ │ │ │ '--no-binary', │ │ │ │ │ ':none:', │ │ │ │ │ ... +9 │ │ │ │ ] │ │ │ │ command_desc = 'pip subprocess to install build dependencies' │ │ │ │ cwd = None │ │ │ │ env = { │ │ │ │ │ 'HOSTNAME': '4ffeab1ce008', │ │ │ │ │ 'PYTHON_VERSION': '3.12.0', │ │ │ │ │ 'PWD': '/usr/src/schemachange', │ │ │ │ │ 'OPENBLAS_NUM_THREADS': '1', │ │ │ │ │ 'HOME': '/root', │ │ │ │ │ 'LANG': 'C.UTF-8', │ │ │ │ │ 'GPG_KEY': │ │ │ │ '7169605F62C751356D054A26A821E680E5FA6305', │ │ │ │ │ 'SHLVL': '1', │ │ │ │ │ 'PYTHON_PIP_VERSION': '23.2.1', │ │ │ │ │ 'PYTHON_GET_PIP_SHA256': │ │ │ │ '22b849a10f86f5ddf7ce148ca2a31214504ee6c83ef6268… │ │ │ │ │ ... +5 │ │ │ │ } │ │ │ │ error = <InstallationSubprocessError(reference='subproce… │ │ │ │ message='[green]pip subprocess to install build │ │ │ │ dependencies[/] did not run successfully.\nexit │ │ │ │ code: 1', context='See above for output.', │ │ │ │ note_stmt='This error originates from a │ │ │ │ subprocess, and is likely not a problem with │ │ │ │ pip.', hint_stmt=None)> │ │ │ │ extra_environ = { │ │ │ │ │ '_PIP_STANDALONE_CERT': │ │ │ │ '/usr/local/lib/python3.12/site-packages/pip/_ve… │ │ │ │ } │ │ │ │ extra_ok_returncodes = [] │ │ │ │ line = '' │ │ │ │ log_failed_cmd = True │ │ │ │ log_subprocess = <bound method VerboseLogger.verbose of │ │ │ │ <VerboseLogger pip.subprocessor (INFO)>> │ │ │ │ on_returncode = 'raise' │ │ │ │ output = 'Collecting setuptools>=40.6.0\n Obtaining │ │ │ │ dependency information for setuptools>'+30381 │ │ │ │ proc = <Popen: returncode: 1 args: │ │ │ │ ['/usr/local/bin/python', │ │ │ │ '/usr/local/lib/python...> │ │ │ │ proc_had_error = True │ │ │ │ show_stdout = False │ │ │ │ showing_subprocess = False │ │ │ │ spinner = <pip._internal.cli.spinners.NonInteractiveSpinner │ │ │ │ object at 0x7f3ae9b0b590> │ │ │ │ stdout_only = False │ │ │ │ unset_environ = [] │ │ │ │ use_spinner = True │ │ │ │ used_level = 15 │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ ╰──────────────────────────────────────────────────────────────────────────────╯ InstallationSubprocessError: pip subprocess to install build dependencies exited with 1 Cleaning up project directory and file based variables 00:01 ERROR: Job failed: exit code 1
@binitbhaskar, It does not appear to be a bug with schemachange.
Schemachange is supported on python 3.8, 3.9 and 3.10. Your docker container seems to be running Python 3.12.
Can you use python 3.10 instead of python 3.12 in your gitlab pipeline?
