Gengliang Wang

Thanks for adding the new function! Merging to master.

@grundprinzip Shall we create a new jira for this instead of using https://issues.apache.org/jira/browse/SPARK-41794?

> BINDING The view will become invalid if the query column-list changes except for the following conditions: The column-list includes a star clause, and there are additional columns. These additional...

@dongjoon-hyun I like the idea. In the long term, the major concern is that there can be issues if a protobuf serializer is not implemented properly in handling null string...

> In my impression, there should be some data models related to history that do not support PROTOBUF? Yes, most of them are minor and won't make too much difference...

@steven-aerts @xkrogen Sorry for the late reply. I will take another look later today.

cc @bozhang2820 since you made https://github.com/apache/spark/commit/551b504cfe38d1ab583e617c37e49659edd65c2e

Merging to master/3.4. cc Spark 3.4.0 release manager @xinrong-meng

@kannankvsp could you show us the usage code and the error stack trace?

@FahimMohd Thanks for the feedback. It is recommended to use PySpark 3.4.0 and above which supports Spark Connect. I will improve this by ignoring the import error of Spark Connect.