incubator-gluten icon indicating copy to clipboard operation
incubator-gluten copied to clipboard

[OPPRO-214] Upgrade velox ww33-34

Open zhejiangxiaomai opened this issue 2 years ago • 4 comments

What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

zhejiangxiaomai avatar Jul 27 '22 10:07 zhejiangxiaomai

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[Gluten-${ISSUES_ID}] ${detailed message}

See also:

github-actions[bot] avatar Jul 27 '22 10:07 github-actions[bot]

Hi Zhenhui, looks like a custom format "dwrf" to Substrait is not supported. If running with "spark.read.format("dwrf").load", below error occurs.

Reason: ReaderFactory is not registered for format unknown
Retriable: False
Expression: it != readerFactories().end()
Function: getReaderFactory
File: ../../velox/dwio/common/ReaderFactory.cpp

rui-mo avatar Aug 02 '22 09:08 rui-mo

Hi Zhenhui, looks like a custom format "dwrf" to Substrait is not supported. If running with "spark.read.format("dwrf").load", below error occurs.

Reason: ReaderFactory is not registered for format unknown
Retriable: False
Expression: it != readerFactories().end()
Function: getReaderFactory
File: ../../velox/dwio/common/ReaderFactory.cpp

OK, I will fix this.

zhejiangxiaomai avatar Aug 02 '22 09:08 zhejiangxiaomai

Hi Zhenhui, looks like a custom format "dwrf" to Substrait is not supported. If running with "spark.read.format("dwrf").load", below error occurs.

Reason: ReaderFactory is not registered for format unknown
Retriable: False
Expression: it != readerFactories().end()
Function: getReaderFactory
File: ../../velox/dwio/common/ReaderFactory.cpp

OK, I will fix this.

Thanks, the DWRF format should be added to the proto file by hand since it is not upstreamed to Substrait yet.

rui-mo avatar Aug 02 '22 09:08 rui-mo