seperate plot 4 phases to phase1 and phase234

[EchoAGI]

create plot_disk_pipeline.hpp from plot_disk.hpp. separate phases to phase1 and phase234, in order to fully resource usage in kubernetes. because resource limit to phase1 and phase234 are not the same.

add -h, phase flag to cli.hpp.

add corresponding python bindings to plot_disk_pipeline.hpp, which has two functions "create_plot_disk_phase1" and "create_plot_disk_phase234"

[arvidn]

a few high-level comments:

• It would be much easier to review if you would not make formatting changes in this PR, but propose those separately
• I think we would prefer comments in english
• I think the justification for this change is a bit light. Could you elaborate on "in order to fully resource usage in kubernetes. because resource limit to phase1 and phase234 are not the same."?

[EchoAGI]

a few high-level comments:

• It would be much easier to review if you would not make formatting changes in this PR, but propose those separately
• I think we would prefer comments in english
• I think the justification for this change is a bit light. Could you elaborate on "in order to fully resource usage in kubernetes. because resource limit to phase1 and phase234 are not the same."?

Thanks! We'll improve the changes later... BTW, why not use multiple merge sort, but bucket sort on disk???

[mgraczyk]

@newtalentxp newtalentxp The data being sorted is usually uniformly distributed, so the bucket sort performs better at the cost of higher memory. It is O(n) instead of O(n logn).

The quicksort_last sort strategy is used to sort the buckets that are not uniformly distributed. A merge sort would probably perform better there. I use std::sort for my own plotting, which in my libstdc++ does an introsort.

[mgraczyk]

IMO the it would be better to first add checkpoints which allow phases to be resumed from start. Then you can run the processes on separate machines by transferring the checkpoint data from machine to machine (or just storing it on a shared location in the first place).

This is pretty easy to do at the beginning and end of each phase.

[github-actions[bot]]

'This PR has been flagged as stale due to no activity for over 60 days. It will not be automatically closed, but it has been given a stale-pr label and should be manually reviewed.'