datafusion-comet icon indicating copy to clipboard operation
datafusion-comet copied to clipboard

PR build for Linux Java 11 with Spark 3.4 is not running

Open kazuyukitanimura opened this issue 1 year ago • 4 comments

Describe the bug

The github action PR build for Linux Java 11 with Spark 3.4 is not running

Steps to reproduce

Looks like the variable substitution is not working, i.e. PR Build / ${{ matrix.os }}/java ${{ matrix.java_version }}-spark-${{ matrix.spark-version }}/${{

Expected behavior

We should run tests for Java 11

Additional context

No response

kazuyukitanimura avatar May 06 '24 17:05 kazuyukitanimura

Screenshot 2024-05-06 at 10 30 18 AM

kazuyukitanimura avatar May 06 '24 17:05 kazuyukitanimura

Java 11 is excluded on purpose on pull request event as there are already too much combinations. Java8 and Java17 with Spark 3.4 on Linux are tested, which should be sufficient? They are all tested in a post-commit event though.

advancedxy avatar May 21 '24 02:05 advancedxy

Thanks @advancedxy Do you know the original reason why this specific combination was excluded by any chance?

kazuyukitanimura avatar May 21 '24 23:05 kazuyukitanimura

There's quota limit per repo for github runners(might not be a problem for apache project but for the forked ones) and we thought it would be sufficient to cover Java8(oldest) and Java17(quite the latest) with Spark 3.4 on Linux runners for pull requests. In that way, the runners required to run PR actions is reduced. The power consumption is reduced as well😁.

Is there any specific need to run Java11, such as it's the java version used in your production? If so, I think we can replace java 17/8 with java11.

advancedxy avatar May 22 '24 02:05 advancedxy

Thanks. Looks like this is the history https://github.com/apache/datafusion-comet/pull/122 We can keep it as is for now. Closing

kazuyukitanimura avatar May 23 '24 05:05 kazuyukitanimura