incubator-wayang icon indicating copy to clipboard operation
incubator-wayang copied to clipboard

remove the set parallelism 1

Open github-actions[bot] opened this issue 2 years ago • 0 comments

remove the set parallelism 1

https://github.com/apache/incubator-wayang/blob/f8692b292d6e988f479699e6c5144fa5d4ba9bf2/wayang-platforms/wayang-flink/code/main/java/org/apache/wayang/flink/operators/FlinkObjectFileSink.java#L79



        assert inputs.length == this.getNumInputs();
        assert outputs.length <= 1;
        final FileChannel.Instance output;
        final String targetPath;
        if(outputs.length == 1) {
            output = (FileChannel.Instance) outputs[0];
            targetPath = output.addGivenOrTempPath(this.textFileUrl, flinkExecutor.getConfiguration());
        }else{
            targetPath = this.textFileUrl;
        }

        //TODO: remove the set parallelism 1
        DataSetChannel.Instance input = (DataSetChannel.Instance) inputs[0];
        final DataSink<Type> tDataSink = input.<Type>provideDataSet()
                .write(new WayangFileOutputFormat<Type>(targetPath), targetPath, FileSystem.WriteMode.OVERWRITE)

034c188dd90f952249452462d3ed4c3af6231d37

github-actions[bot] avatar Mar 18 '22 08:03 github-actions[bot]