arrow
arrow copied to clipboard
apache
ARROW-17175: [CI][macOS] macos-10.15 is deprecated and macos-latest is macos-11
https://github.com/actions/virtual-environments#available-environments
macOS 11 macos-latest or macos-11 macOS 10.15 deprecated macos-10.15
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
:warning: Ticket has not been started in JIRA, please click 'Start Progress'.
Revision: 798b54ef4364226b7e08456e124171dbcaf950dd
Submitted crossbow builds: ursacomputing/crossbow @ actions-e0531f4869
| Task | Status |
|---|---|
| homebrew-cpp |  |
| homebrew-cpp-autobrew |  |
| homebrew-r-autobrew |  |
| homebrew-r-brew |  |
| wheel-macos-big-sur-cp310-arm64 |  |
| wheel-macos-big-sur-cp310-universal2 |  |
| wheel-macos-big-sur-cp38-arm64 |  |
| wheel-macos-big-sur-cp39-arm64 |  |
| wheel-macos-big-sur-cp39-universal2 |  |
| wheel-macos-high-sierra-cp310-amd64 |  |
| wheel-macos-high-sierra-cp37-amd64 |  |
| wheel-macos-high-sierra-cp38-amd64 |  |
| wheel-macos-high-sierra-cp39-amd64 |  |
| wheel-macos-mavericks-cp310-amd64 |  |
| wheel-macos-mavericks-cp37-amd64 |  |
| wheel-macos-mavericks-cp38-amd64 |  |
| wheel-macos-mavericks-cp39-amd64 |  |
Revision: 63f4ad885d66f74556b761704ffcd6af4baddb13
Submitted crossbow builds: ursacomputing/crossbow @ actions-9751fb9f1c
| Task | Status |
|---|---|
| homebrew-cpp |  |
| homebrew-cpp-autobrew |  |
| homebrew-r-autobrew |  |
| homebrew-r-brew |  |
Some of our github action builds for macos 10.15 have been failing today as per the deprecation schedule on this issue: https://github.com/actions/virtual-environments/issues/5583
As per this comment https://github.com/actions/virtual-environments/issues/5583#issuecomment-1134941811 macos-latest will be macos 12 after the transition.
Also will this change be an issue for the wheel builds? I know we set 10.15 as the deployment target anyway but afaik it would still be better to test on 10.15? There was some discussion about this in regards to the self-hosted runners in crossbow @pitrou @jorisvandenbossche
@jeroen @nealrichardson Should we keep CI jobs for https://github.com/autobrew/homebrew-core/blob/high-sierra/Formula/apache-arrow.rb ? It's for https://github.com/autobrew/brew not https://github.com/Homebrew/brew but we use https://github.com/Homebrew/brew to test https://github.com/apache/arrow/blob/master/dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb .
FYI: https://github.com/Homebrew/brew removed cellar in bottle block ( https://github.com/Homebrew/brew/commit/f65d525693a45c8e4c2ebc1452080f08c363ee15 ) but https://github.com/autobrew/homebrew-core/blob/high-sierra/Formula/apache-arrow.rb and https://github.com/apache/arrow/blob/master/dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb still use cellar because they are for https://github.com/autobrew/brew not https://github.com/Homebrew/brew .
If we need to keep CI jobs for https://github.com/autobrew/homebrew-core/blob/high-sierra/Formula/apache-arrow.rb, we need to replace Homebrew installed in the macos-latest GitHub Actions runner with https://github.com/autobrew/brew . (I don't know how to do it.)
It seems that https://github.com/autobrew/homebrew-cran is based on https://github.com/Homebrew/brew . Can we remove CI jobs for https://github.com/autobrew/homebrew-core/blob/high-sierra/Formula/apache-arrow.rb and add new CI jobs for https://github.com/autobrew/homebrew-cran/blob/master/Formula/apache-arrow-static.rb (as a follow-up task)?
Autobrew is maintained by Jeroen Ooms and is necessary for the R package due to CRAN targeting EOL macos versions (see here). So removing the autobrew formula is not an option. I don't think homebrew-cpp-autobrew is used for anything so removing that should be fine.
IIUC the version manipulation in utils-prepare.sh is necessary due to the possibility that the R package adds a 4th version component for R-only fixes during CRAN submission e.g. 6.0.0.2.
Thanks for sharing about the context. It seems that we can't run CI jobs for autobrew because we don't have CI environment for old macOS. So can we remove CI jobs for autobrew?
IIUC the version manipulation in
utils-prepare.shis necessary due to the possibility that the R package adds a 4th version component for R-only fixes during CRAN submission e.g. 6.0.0.2.
The removed codes in utils-prepare.sh are only for autobrew's CI jobs. It's not related to release process. If we remove autobrew's CI jobs, we can remove them safely.
Revision: a011390d6bd671a461a50ae23f65eed1b4189d3c
Submitted crossbow builds: ursacomputing/crossbow @ actions-3dbd7f7c62
| Task | Status |
|---|---|
| wheel-macos-big-sur-cp310-arm64 |  |
| wheel-macos-big-sur-cp310-universal2 |  |
| wheel-macos-big-sur-cp38-arm64 |  |
| wheel-macos-big-sur-cp39-arm64 |  |
| wheel-macos-big-sur-cp39-universal2 |  |
| wheel-macos-high-sierra-cp310-amd64 |  |
| wheel-macos-high-sierra-cp37-amd64 |  |
| wheel-macos-high-sierra-cp38-amd64 |  |
| wheel-macos-high-sierra-cp39-amd64 |  |
| wheel-macos-mavericks-cp310-amd64 |  |
| wheel-macos-mavericks-cp37-amd64 |  |
| wheel-macos-mavericks-cp38-amd64 |  |
| wheel-macos-mavericks-cp39-amd64 |  |
we don't have CI environment for old macOS
We are working on 10.13 self-hosted runners specifically for the R auto-brew builds atm. As CRAN builds on 10.13 and will block our submission if we don't pass on it, we really want to make sure it works to avoid lengthy discussions with CRAN. So please do not remove those builds.
What are the issues you had with the self-hosted mac runner? We will likely keep it deactivated as we have the new, extended capacity in crossbow that should take care of our mac build needs. (ex. autobrew :) )
We are working on 10.13 self-hosted runners specifically for the R auto-brew builds atm.
Wow! I didn't know it! What runs-on should we use for the 10.13 self-hosted runners?
Due to the 10.13 being eol homebrew does not work so we won't be able to use the normal R install actions but rather will have to customize the templates a bit. We will likely have separate tags for r-oldrel and r-release version (so maybe macos-13-release ?).
For now I would think it best/easiest to comment these tasks out from tasks.yml and update the workflows & tags in a follow up.
There is also https://issues.apache.org/jira/browse/ARROW-12590 which probably needs to happen now as the old formula is what is causing the issue on macos >= 11. I'll check it out
We could than add a param/matrix for the runner so that we can run this for both 10.13 and >=11
The wheel failure is a pip installation failure of crc32c, which it now tries to install from source instead of using a wheel
The wheel failure is a pip installation failure of crc32c, which it now tries to install from source instead of using a wheel
Which is weird, because binary wheels are provided. Perhaps need to update pip to the latest version first?
It looks like the azure builds will also be affected https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=30742&view=logs&j=cf796865-97b7-5cd1-be8e-6e00ce4fd8cf
Revision: 288fba4485d8efdccf7dd360cf775c1c79eb7f91
Submitted crossbow builds: ursacomputing/crossbow @ actions-389e663b84
| Task | Status |
|---|---|
| wheel-macos-big-sur-cp310-arm64 |  |
| wheel-macos-big-sur-cp310-universal2 |  |
| wheel-macos-big-sur-cp38-arm64 |  |
| wheel-macos-big-sur-cp39-arm64 |  |
| wheel-macos-big-sur-cp39-universal2 |  |
| wheel-macos-high-sierra-cp310-amd64 |  |
| wheel-macos-high-sierra-cp37-amd64 |  |
| wheel-macos-high-sierra-cp38-amd64 |  |
| wheel-macos-high-sierra-cp39-amd64 |  |
| wheel-macos-mavericks-cp310-amd64 |  |
| wheel-macos-mavericks-cp37-amd64 |  |
| wheel-macos-mavericks-cp38-amd64 |  |
| wheel-macos-mavericks-cp39-amd64 |  |
Revision: 288fba4485d8efdccf7dd360cf775c1c79eb7f91
Submitted crossbow builds: ursacomputing/crossbow @ actions-85172d3456
| Task | Status |
|---|---|
| conda-osx-arm64-clang-py310 | |
| conda-osx-arm64-clang-py38 | |
| conda-osx-arm64-clang-py39 | |
| conda-osx-clang-py310 | |
| conda-osx-clang-py37-r40 | |
| conda-osx-clang-py37-r41 | |
| conda-osx-clang-py38 | |
| conda-osx-clang-py39 |
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
Revision: 05c717d36bd71675018cacafede39908458ecc41
Submitted crossbow builds: ursacomputing/crossbow @ actions-7896db46bb
| Task | Status |
|---|---|
| almalinux-8-amd64 |  |
| almalinux-8-arm64 |  |
| almalinux-9-amd64 |  |
| almalinux-9-arm64 |  |
| amazon-linux-2-amd64 |  |
| amazon-linux-2-arm64 |  |
| centos-7-amd64 |  |
| centos-8-stream-amd64 |  |
| centos-8-stream-arm64 |  |
| centos-9-stream-amd64 |  |
| centos-9-stream-arm64 |  |
| conan-maximum |  |
| conan-minimum |  |
| conda-clean | |
| conda-linux-gcc-py310-arm64 | |
| conda-linux-gcc-py310-cpu | |
| conda-linux-gcc-py310-cuda | |
| conda-linux-gcc-py310-ppc64le | |
| conda-linux-gcc-py37-arm64 | |
| conda-linux-gcc-py37-cpu-r40 | |
| conda-linux-gcc-py37-cpu-r41 | |
| conda-linux-gcc-py37-cuda | |
| conda-linux-gcc-py37-ppc64le | |
| conda-linux-gcc-py38-arm64 | |
| conda-linux-gcc-py38-cpu | |
| conda-linux-gcc-py38-cuda | |
| conda-linux-gcc-py38-ppc64le | |
| conda-linux-gcc-py39-arm64 | |
| conda-linux-gcc-py39-cpu | |
| conda-linux-gcc-py39-cuda | |
| conda-linux-gcc-py39-ppc64le | |
| conda-osx-arm64-clang-py310 | |
| conda-osx-arm64-clang-py38 | |
| conda-osx-arm64-clang-py39 | |
| conda-osx-clang-py310 | |
| conda-osx-clang-py37-r40 | |
| conda-osx-clang-py37-r41 | |
| conda-osx-clang-py38 | |
| conda-osx-clang-py39 | |
| conda-win-vs2017-py310 | |
| conda-win-vs2017-py37-r40 | |
| conda-win-vs2017-py37-r41 | |
| conda-win-vs2017-py38 | |
| conda-win-vs2017-py39 | |
| debian-bookworm-amd64 |  |
| debian-bookworm-arm64 |  |
| debian-bullseye-amd64 |  |
| debian-bullseye-arm64 |  |
| example-cpp-minimal-build-static |  |
| example-cpp-minimal-build-static-system-dependency |  |
| example-python-minimal-build-fedora-conda |  |
| example-python-minimal-build-ubuntu-venv |  |
| homebrew-cpp |  |
| homebrew-r-autobrew |  |
| homebrew-r-brew |  |
| java-jars |  |
| nuget |  |
| python-sdist |  |
| r-binary-packages |  |
| test-build-cpp-fuzz |  |
| test-build-vcpkg-win |  |
| test-conda-cpp |  |
| test-conda-cpp-valgrind | |
| test-conda-python-3.10 |  |
| test-conda-python-3.7 |  |
| test-conda-python-3.7-hdfs-2.9.2 |  |
| test-conda-python-3.7-hdfs-3.2.1 |  |
| test-conda-python-3.7-kartothek-latest |  |
| test-conda-python-3.7-kartothek-master |  |
| test-conda-python-3.7-pandas-0.24 |  |
| test-conda-python-3.7-pandas-latest |  |
| test-conda-python-3.7-spark-v3.1.2 |  |
| test-conda-python-3.8 |  |
| test-conda-python-3.8-hypothesis |  |
| test-conda-python-3.8-pandas-latest |  |
| test-conda-python-3.8-pandas-nightly |  |
| test-conda-python-3.8-spark-v3.2.0 |  |
| test-conda-python-3.9 |  |
| test-conda-python-3.9-dask-latest |  |
| test-conda-python-3.9-dask-master |  |
| test-conda-python-3.9-pandas-master |  |
| test-conda-python-3.9-spark-master |  |
| test-debian-10-cpp-amd64 |  |
| test-debian-10-cpp-i386 |  |
| test-debian-11-cpp-amd64 |  |
| test-debian-11-cpp-i386 |  |
| test-debian-11-go-1.16 | |
| test-debian-11-python-3 | |
| test-debian-c-glib |  |
| test-debian-ruby |  |
| test-fedora-35-cpp |  |
| test-fedora-35-python-3 | |
| test-fedora-r-clang-sanitizer | |
| test-r-arrow-backwards-compatibility |  |
| test-r-depsource-bundled | |
| test-r-depsource-system |  |
| test-r-dev-duckdb |  |
| test-r-devdocs |  |
| test-r-gcc-11 |  |
| test-r-gcc-12 |  |
| test-r-install-local |  |
| test-r-linux-as-cran |  |
| test-r-linux-rchk |  |
| test-r-linux-valgrind | |
| test-r-minimal-build | |
| test-r-offline-maximal |  |
| test-r-offline-minimal | |
| test-r-rhub-debian-gcc-devel-lto-latest | |
| test-r-rhub-debian-gcc-release-custom-ccache | |
| test-r-rhub-ubuntu-gcc-release-latest | |
| test-r-rocker-r-base-latest | |
| test-r-rstudio-r-base-4.1-opensuse153 | |
| test-r-rstudio-r-base-4.2-centos7-devtoolset-8 | |
| test-r-rstudio-r-base-4.2-focal | |
| test-r-ubuntu-22.04 |  |
| test-r-versions |  |
| test-skyhook-integration |  |
| test-ubuntu-18.04-cpp |  |
| test-ubuntu-18.04-cpp-release |  |
| test-ubuntu-18.04-cpp-static |  |
| test-ubuntu-18.04-r-sanitizer | |
| test-ubuntu-20.04-cpp |  |
| test-ubuntu-20.04-cpp-14 |  |
| test-ubuntu-20.04-cpp-17 |  |
| test-ubuntu-20.04-cpp-bundled |  |
| test-ubuntu-20.04-cpp-thread-sanitizer |  |
| test-ubuntu-20.04-python-3 | |
| test-ubuntu-22.04-cpp |  |
| test-ubuntu-c-glib |  |
| test-ubuntu-default-docs | |
| test-ubuntu-ruby |  |
| ubuntu-bionic-amd64 |  |
| ubuntu-bionic-arm64 |  |
| ubuntu-focal-amd64 |  |
| ubuntu-focal-arm64 |  |
| ubuntu-jammy-amd64 |  |
| ubuntu-jammy-arm64 |  |
| verify-rc-source-cpp-linux-almalinux-8-amd64 |  |
| verify-rc-source-cpp-linux-conda-latest-amd64 |  |
| verify-rc-source-cpp-linux-ubuntu-18.04-amd64 |  |
| verify-rc-source-cpp-linux-ubuntu-20.04-amd64 |  |
| verify-rc-source-cpp-linux-ubuntu-22.04-amd64 |  |
| verify-rc-source-cpp-macos-amd64 |  |
| verify-rc-source-cpp-macos-arm64 |  |
| verify-rc-source-cpp-macos-conda-amd64 |  |
| verify-rc-source-csharp-linux-almalinux-8-amd64 |  |
| verify-rc-source-csharp-linux-conda-latest-amd64 |  |
| verify-rc-source-csharp-linux-ubuntu-18.04-amd64 |  |
| verify-rc-source-csharp-linux-ubuntu-20.04-amd64 |  |
| verify-rc-source-csharp-linux-ubuntu-22.04-amd64 |  |
| verify-rc-source-csharp-macos-amd64 |  |
| verify-rc-source-csharp-macos-arm64 |  |
| verify-rc-source-go-linux-almalinux-8-amd64 |  |
| verify-rc-source-go-linux-conda-latest-amd64 |  |
| verify-rc-source-go-linux-ubuntu-18.04-amd64 |  |
| verify-rc-source-go-linux-ubuntu-20.04-amd64 |  |
| verify-rc-source-go-linux-ubuntu-22.04-amd64 |  |
| verify-rc-source-go-macos-amd64 |  |
| verify-rc-source-go-macos-arm64 |  |
| verify-rc-source-integration-linux-almalinux-8-amd64 |  |
| verify-rc-source-integration-linux-conda-latest-amd64 |  |
| verify-rc-source-integration-linux-ubuntu-18.04-amd64 |  |
| verify-rc-source-integration-linux-ubuntu-20.04-amd64 |  |
| verify-rc-source-integration-linux-ubuntu-22.04-amd64 |  |
| verify-rc-source-integration-macos-amd64 |  |
| verify-rc-source-integration-macos-arm64 |  |
| verify-rc-source-integration-macos-conda-amd64 |  |
| verify-rc-source-java-linux-almalinux-8-amd64 |  |
| verify-rc-source-java-linux-conda-latest-amd64 |  |
| verify-rc-source-java-linux-ubuntu-18.04-amd64 |  |
| verify-rc-source-java-linux-ubuntu-20.04-amd64 |  |
| verify-rc-source-java-linux-ubuntu-22.04-amd64 |  |
| verify-rc-source-java-macos-amd64 |  |
| verify-rc-source-js-linux-almalinux-8-amd64 |  |
| verify-rc-source-js-linux-conda-latest-amd64 |  |
| verify-rc-source-js-linux-ubuntu-18.04-amd64 |  |
| verify-rc-source-js-linux-ubuntu-20.04-amd64 |  |
| verify-rc-source-js-linux-ubuntu-22.04-amd64 |  |
| verify-rc-source-js-macos-amd64 |  |
| verify-rc-source-js-macos-arm64 |  |
| verify-rc-source-python-linux-almalinux-8-amd64 |  |
| verify-rc-source-python-linux-conda-latest-amd64 |  |
| verify-rc-source-python-linux-ubuntu-18.04-amd64 |  |
| verify-rc-source-python-linux-ubuntu-20.04-amd64 |  |
| verify-rc-source-python-linux-ubuntu-22.04-amd64 |  |
| verify-rc-source-python-macos-amd64 |  |
| verify-rc-source-python-macos-arm64 |  |
| verify-rc-source-python-macos-conda-amd64 |  |
| verify-rc-source-ruby-linux-almalinux-8-amd64 |  |
| verify-rc-source-ruby-linux-conda-latest-amd64 |  |
| verify-rc-source-ruby-linux-ubuntu-18.04-amd64 |  |
| verify-rc-source-ruby-linux-ubuntu-20.04-amd64 |  |
| verify-rc-source-ruby-linux-ubuntu-22.04-amd64 |  |
| verify-rc-source-ruby-macos-amd64 |  |
| verify-rc-source-ruby-macos-arm64 |  |
| verify-rc-source-windows |  |
| wheel-macos-big-sur-cp310-arm64 |  |
| wheel-macos-big-sur-cp310-universal2 |  |
| wheel-macos-big-sur-cp38-arm64 |  |
| wheel-macos-big-sur-cp39-arm64 |  |
| wheel-macos-big-sur-cp39-universal2 |  |
| wheel-macos-high-sierra-cp310-amd64 |  |
| wheel-macos-high-sierra-cp37-amd64 |  |
| wheel-macos-high-sierra-cp38-amd64 |  |
| wheel-macos-high-sierra-cp39-amd64 |  |
| wheel-macos-mavericks-cp310-amd64 |  |
| wheel-macos-mavericks-cp37-amd64 |  |
| wheel-macos-mavericks-cp38-amd64 |  |
| wheel-macos-mavericks-cp39-amd64 |  |
| wheel-manylinux2014-cp310-amd64 |  |
| wheel-manylinux2014-cp310-arm64 |  |
| wheel-manylinux2014-cp37-amd64 |  |
| wheel-manylinux2014-cp37-arm64 |  |
| wheel-manylinux2014-cp38-amd64 |  |
| wheel-manylinux2014-cp38-arm64 |  |
| wheel-manylinux2014-cp39-amd64 |  |
| wheel-manylinux2014-cp39-arm64 |  |
| wheel-windows-cp310-amd64 |  |
| wheel-windows-cp37-amd64 |  |
| wheel-windows-cp38-amd64 |  |
| wheel-windows-cp39-amd64 |  |
Revision: 05c717d36bd71675018cacafede39908458ecc41
Submitted crossbow builds: ursacomputing/crossbow @ actions-8e344ac9dc
| Task | Status |
|---|---|
| verify-rc-source-cpp-macos-amd64 |  |
| verify-rc-source-cpp-macos-arm64 |  |
| verify-rc-source-cpp-macos-conda-amd64 |  |
| verify-rc-source-csharp-macos-amd64 |  |
| verify-rc-source-csharp-macos-arm64 |  |
| verify-rc-source-go-macos-amd64 |  |
| verify-rc-source-go-macos-arm64 |  |
| verify-rc-source-integration-macos-amd64 |  |
| verify-rc-source-integration-macos-arm64 |  |
| verify-rc-source-integration-macos-conda-amd64 |  |
| verify-rc-source-java-macos-amd64 |  |
| verify-rc-source-js-macos-amd64 |  |
| verify-rc-source-js-macos-arm64 |  |
| verify-rc-source-python-macos-amd64 |  |
| verify-rc-source-python-macos-arm64 |  |
| verify-rc-source-python-macos-conda-amd64 |  |
| verify-rc-source-ruby-macos-amd64 |  |
| verify-rc-source-ruby-macos-arm64 |  |