roryqi

JVM can occupy the memory although they don't process any data.

Why do the shuffle server restart? There should be some information in the logs or stdout.

> It is restart by k8s, reason is memory is too high.I think if memory release ,it will not be appear Maybe we should give more memory to the pod.

You can adjust the parameter of memory in the `bin/rss-env.sh` and conf/server.conf.

> XMX_SIZE ? it is 30G now. Could you reduce the value?

> I have reduced it to 8G，but the pod restart too Does the server restart because of the same reason? You give the pod 32G memory, XMX_SIZE is 8G, don't...

It's better to sort MapId before the data are flushed.It won't bring too much cost for non-AQE optimized stages.

> > It's better to sort MapId before the data are flushed.It won't bring too much cost for non-AQE optimized stages. > > Does data need to sort by mapId?...

> > > > It's better to sort MapId before the data are flushed.It won't bring too much cost for non-AQE optimized stages. > > > > > > >...

> > taskId-1 block , taskId-2 block, taskId-3 block, taskId-1 block, taskId-2 block, taskId-6 block. > > If one reader want the data from taskId=1, so it still want to...