arrow icon indicating copy to clipboard operation
arrow copied to clipboard

ARROW-17817: [C++] Let ORC compile on MSVC if it is activated

Open LouisClt opened this issue 2 years ago • 8 comments

LouisClt avatar Sep 22 '22 14:09 LouisClt

Could you enable ORC in our CI jobs?

diff --git a/.github/workflows/cpp.yml b/.github/workflows/cpp.yml
index 2642a6ec1a..07243ffc4e 100644
--- a/.github/workflows/cpp.yml
+++ b/.github/workflows/cpp.yml
@@ -213,6 +213,7 @@ jobs:
       ARROW_HOME: /usr
       ARROW_JEMALLOC: OFF
       ARROW_MIMALLOC: ON
+      ARROW_ORC: ON
       ARROW_PARQUET: ON
       ARROW_USE_GLOG: OFF
       ARROW_VERBOSE_THIRDPARTY_BUILD: OFF
diff --git a/ci/appveyor-cpp-build.bat b/ci/appveyor-cpp-build.bat
index 4b90e49257..d8eb9d2c11 100644
--- a/ci/appveyor-cpp-build.bat
+++ b/ci/appveyor-cpp-build.bat
@@ -68,6 +68,7 @@ cmake -G "%GENERATOR%" %CMAKE_ARGS% ^
       -DARROW_FLIGHT_SQL=%ARROW_BUILD_FLIGHT_SQL% ^
       -DARROW_GANDIVA=%ARROW_BUILD_GANDIVA% ^
       -DARROW_MIMALLOC=ON ^
+      -DARROW_ORC=ON ^
       -DARROW_PARQUET=ON ^
       -DARROW_PYTHON=ON ^
       -DARROW_S3=%ARROW_S3% ^

kou avatar Sep 22 '22 20:09 kou

https://issues.apache.org/jira/browse/ARROW-17817

github-actions[bot] avatar Sep 22 '22 21:09 github-actions[bot]

:warning: Ticket has not been started in JIRA, please click 'Start Progress'.

github-actions[bot] avatar Sep 22 '22 21:09 github-actions[bot]

Hello, I made the changes you suggested. I don't know if this will compile fine as my only compile tests were with vcpkg.

LouisClt avatar Sep 23 '22 07:09 LouisClt

I can try to fix them, at least for the easy ones. How's that these warnings did not fire before ? It is specific to Visual C++ ?

LouisClt avatar Sep 23 '22 19:09 LouisClt

I think so.

kou avatar Sep 23 '22 19:09 kou

Could you apply this for the AppVeyor job failure?

diff --git a/ci/conda_env_cpp.txt b/ci/conda_env_cpp.txt
index dd313f19d7..40871f560d 100644
--- a/ci/conda_env_cpp.txt
+++ b/ci/conda_env_cpp.txt
@@ -37,6 +37,7 @@ ninja
 # Required by google-cloud-cpp, the Conda package is missing the dependency:
 #    https://github.com/conda-forge/google-cloud-cpp-feedstock/issues/28
 nlohmann_json
+orc
 pkg-config
 python
 rapidjson

https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/44867881#L740

CMake Error at cmake_modules/FindORC.cmake:56 (message):
  ORC library was required in toolchain and unable to locate
Call Stack (most recent call first):
  cmake_modules/ThirdpartyToolchain.cmake:280 (find_package)
  cmake_modules/ThirdpartyToolchain.cmake:4339 (resolve_dependency)
  CMakeLists.txt:579 (include)

kou avatar Sep 23 '22 20:09 kou

I made the changes for the AppVeyor job failure. However this still does not work because it can't find a correct version of protobuf (from looking at the logs). There is also a problem in the "C++ / AMD64 Windows 2019 C++17" job, it seems it can't find some Zstd symbols during linking.

LouisClt avatar Sep 27 '22 07:09 LouisClt

I've fixed CI failures on AppVeyor and GitHub Actions.

I'll merge this tomorrow if nobody objects it.

kou avatar Oct 13 '22 03:10 kou

@github-actions crossbow submit java-jars

kou avatar Oct 14 '22 14:10 kou

Revision: f5ede2c962514c6f867f9a83adee755ec7a768b6

Submitted crossbow builds: ursacomputing/crossbow @ actions-5f0a218feb

Task Status
java-jars Github Actions

github-actions[bot] avatar Oct 14 '22 17:10 github-actions[bot]

@github-actions crossbow submit java-jars

kou avatar Oct 14 '22 21:10 kou

Revision: b7b3f6e4efcddabec55b682c58cb36d2dc501234

Submitted crossbow builds: ursacomputing/crossbow @ actions-e1f7effb08

Task Status
java-jars Github Actions

github-actions[bot] avatar Oct 14 '22 21:10 github-actions[bot]

Benchmark runs are scheduled for baseline = f0cf5c2033b99897d31c297f2cc116fa2689feec and contender = aeba61663fdd82719e6cc0945aba216958ad6970. aeba61663fdd82719e6cc0945aba216958ad6970 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. Conbench compare runs links: [Finished :arrow_down:0.0% :arrow_up:0.0%] ec2-t3-xlarge-us-east-2 [Failed :arrow_down:0.56% :arrow_up:0.0%] test-mac-arm [Finished :arrow_down:0.0% :arrow_up:0.0%] ursa-i9-9960x [Finished :arrow_down:0.61% :arrow_up:0.04%] ursa-thinkcentre-m75q Buildkite builds: [Finished] aeba6166 ec2-t3-xlarge-us-east-2 [Failed] aeba6166 test-mac-arm [Finished] aeba6166 ursa-i9-9960x [Finished] aeba6166 ursa-thinkcentre-m75q [Finished] f0cf5c20 ec2-t3-xlarge-us-east-2 [Failed] f0cf5c20 test-mac-arm [Finished] f0cf5c20 ursa-i9-9960x [Finished] f0cf5c20 ursa-thinkcentre-m75q Supported benchmarks: ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True test-mac-arm: Supported benchmark langs: C++, Python, R ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

ursabot avatar Oct 17 '22 15:10 ursabot

['Python', 'R'] benchmarks have high level of regressions. test-mac-arm ursa-i9-9960x

ursabot avatar Oct 17 '22 15:10 ursabot