incubator-wayang
incubator-wayang copied to clipboard
apache
Apache Wayang(incubating) is the first cross-platform data processing system.
download Apache rat if is needed https://github.com/apache/incubator-wayang/blob/4cc0bfdca06dda171b661822daae5fa438d9d475/python/bin/check-license.sh#L22 ```sh #!/bin/bash ################################################################################ ## ## Licensed to the Apache Software Foundation (ASF) under one or more ## contributor license agreements. See the NOTICE...
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "github-actions[bot] avatar"){kind=avatar}
The [platform-flink](https://github.com/apache/incubator-wayang/tree/main/wayang-platforms/wayang-flink/code) was added and had just the integrations test, but it does not have the operator test like in [platform-spark](https://github.com/apache/incubator-wayang/tree/main/wayang-platforms/wayang-spark/code/test/java/org/apache/wayang/spark) The idea is to create a similar operators test...
It is unnecessary that by default in TEST Mode, there are displayed as a result all the Logs from Flink, Spark, etc. This should be first configured by the developer.
remove the set parallelism 1 https://github.com/apache/incubator-wayang/blob/f8692b292d6e988f479699e6c5144fa5d4ba9bf2/wayang-platforms/wayang-flink/code/main/java/org/apache/wayang/flink/operators/FlinkObjectFileSink.java#L79 ```java assert inputs.length == this.getNumInputs(); assert outputs.length
validate if the implementation apply for the case final Optional fileSystem = FileSystems.getFileSystem(ObjectFileSource.this.inputUrl); if (fileSystem.isPresent()) { final int KiB = 1024; final int MiB = 1024 * KiB; try (LimitedInputStream...
### Description I'm trying to run TPC-H Q3 and compare the performance between Wayang and SparkSQL under the following setup: * Running both Spark (3.5.1) and Wayang on a local...
Is there a way to explain (print out) the logical and physical plan for wayang?
I tried to run the Wordcount with Flink and got this error: Exception in thread "main" org.apache.wayang.core.api.exception.WayangException: Executing T[FlinkReduceBy[Add counters]] failed. at org.apache.wayang.flink.execution.FlinkExecutor.execute(FlinkExecutor.java:113) at org.apache.wayang.core.platform.PushExecutorTemplate.execute(PushExecutorTemplate.java:73) at org.apache.wayang.core.platform.PushExecutorTemplate$StageExecution.execute(PushExecutorTemplate.java:195) at org.apache.wayang.core.platform.PushExecutorTemplate$StageExecution.doExecute(PushExecutorTemplate.java:166) at...
The jobs in the wayang-benchmark module need to be rewritten/cleaned: either because they are using the foundational java plan building way (the Java ones) or because they are using some...