GraphScope
GraphScope copied to clipboard
Reduce memory usage during graph construction
The graph construction requires a reasonably large amount of memory compared to the file size, maybe there is some room for optimization. For example, do some profiling and optimization in following scenario:
- [ ] Multiple labels case, like in ldbc, which will have many unnecessary offset arrays.
- [ ] Large vertex amount case, which will result in large global vertex map.
- [ ] The runtime peak memory usage is 1.5 times of the final graph size, due to parallel sorting, etc. See if this could be reduced or give option to reduce the parallelism.
Some other optimization opportunities may came up during profiling.