Kerwin Zhang

> waiting for worker graceful shutdown, wait for 599s > waiting for worker graceful shutdown, wait for 600s > Failed to stop server(pid=6960) after 600s > Error: Process completed with...

> There's one line of error log, is this related? `/opt/celeborn/conf/celeborn-env.sh: line 1: -e: command not found` @PHILO-HE @zhouyuan This exception seems to be a problem with the command written...

> @kerwin-zk Could you help to check whether this patch could work? Besides, I noticed it requires some extra configurations to be set for celeborn. Here's what I found and...

> --conf spark.shuffle.manager=celeborn \ > --conf spark.celeborn.client.spark.shuffle.writer=sort @FelixYBW @marin-ma I haven't tested row-based sort + Celeborn yet. For columnar-based sort + Celeborn, the following settings are needed: ``` spark.shuffle.manager: org.apache.spark.shuffle.gluten.celeborn.CelebornShuffleManager...

@marin-ma Since our internal version depends on GLUTEN_RSS_SORT_SHUFFLE_WRITER, I suggest adding a configuration to control whether to use GLUTEN_SORT_SHUFFLE_WRITER or GLUTEN_RSS_SORT_SHUFFLE_WRITER.

> > @marin-ma Since our internal version depends on GLUTEN_RSS_SORT_SHUFFLE_WRITER, I suggest adding a configuration to control whether to use GLUTEN_SORT_SHUFFLE_WRITER or GLUTEN_RSS_SORT_SHUFFLE_WRITER. > > Is your internal version the...