Xiaoxuan

Some code refactors to support hash join spill in followup

11

- Add to expose some fields in hash function which will be used for recursive hash control later - Refactor HashBuild a bit by putting hash table build logic into...

Add union batch reader and refactor spill stream objects

5

- Refactor spill stream related object structure: (1) Rename SpillStream to SpillMergeStream as we now support BatchStream for non-sort use case as hash join. (2) SpillFile doesn't implement SpillStream interface...

disk spilling support for hash probe

4

The added hash probe procedure works as follow: 1. hash probe operator waits for table built by hash build operators to join 2. hash probe operator prepare for spilling: (1)...

Add check on the memory usage is zero on pool/tracker destruction

3

Add check on the memory usages are zero on pool/tracker destruction

Update coding style with test method annotation

1

Update coding style with test method annotation, e.g. if we want to add a test method say 'foo' to an object, we will name it with prefix TEST, TEST_foo.

Fix probe spill check failure

3

The maybeReadSpillOutput has a bug that reset spillInputReader_ but not spillOutputReader_ during spill output processing which caused the wrong probe finish condition to be detected. The hash probe operator finishes...

Fix recover ssd cache data by preserving file id map

2

In Meta interactive Prestissimo workload, we found there is not much read can hit in SSD cache after cluster upgrade or worker reboot if we rerun the same set of...