perf: rework scheduler

[westonpace]

A work in progress potential scheduler rework. I think it'll need to wait until we experiment with I/O uring to see if it helps or not. Oddly it does not help improve performance of the scheduler benchmark but makes a significant difference in the file reader benchmark but only when using disk.

[westonpace]

Take benchmark:

reader_2.0_full_disk/take
                        time:   [34.216 µs 34.566 µs 35.117 µs]
                        thrpt:  [2.8476 Melem/s 2.8930 Melem/s 2.9226 Melem/s]

reader_2.1_full_disk/take
                        time:   [34.778 µs 35.547 µs 36.436 µs]
                        thrpt:  [2.7445 Melem/s 2.8131 Melem/s 2.8754 Melem/s]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

reader_2.0_lite_disk/take
                        time:   [27.609 µs 27.931 µs 28.546 µs]
                        thrpt:  [3.5032 Melem/s 3.5803 Melem/s 3.6220 Melem/s]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

reader_2.1_lite_disk/take
                        time:   [28.976 µs 29.801 µs 31.247 µs]
                        thrpt:  [3.2003 Melem/s 3.3556 Melem/s 3.4511 Melem/s]

reader_2.0_full_memory/take
                        time:   [27.318 µs 27.942 µs 28.833 µs]
                        thrpt:  [3.4682 Melem/s 3.5788 Melem/s 3.6605 Melem/s]

reader_2.1_full_memory/take
                        time:   [28.219 µs 28.816 µs 29.563 µs]
                        thrpt:  [3.3826 Melem/s 3.4702 Melem/s 3.5437 Melem/s]
Found 2 outliers among 10 measurements (20.00%)
  1 (10.00%) high mild
  1 (10.00%) high severe

reader_2.0_lite_memory/take
                        time:   [26.575 µs 27.542 µs 28.566 µs]
                        thrpt:  [3.5006 Melem/s 3.6308 Melem/s 3.7630 Melem/s]
Found 2 outliers among 10 measurements (20.00%)
  2 (20.00%) high mild

reader_2.1_lite_memory/take
                        time:   [26.724 µs 27.312 µs 27.651 µs]
                        thrpt:  [3.6166 Melem/s 3.6614 Melem/s 3.7420 Melem/s]

[codecov[bot]]

Codecov Report

:x: Patch coverage is 13.08594% with 445 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance-io/src/scheduler/lite.rs	0.00%	385 Missing :warning:
rust/lance-io/src/scheduler.rs	50.54%	45 Missing :warning:
rust/lance-index/src/vector/ivf/shuffler.rs	20.00%	3 Missing and 1 partial :warning:
rust/lance-tools/src/meta.rs	0.00%	3 Missing :warning:
rust/lance/src/dataset/fragment.rs	40.00%	2 Missing and 1 partial :warning:
rust/lance/src/dataset/statistics.rs	0.00%	2 Missing :warning:
rust/lance/src/index/vector/ivf/v2.rs	77.77%	1 Missing and 1 partial :warning:
rust/lance/src/io/exec/scan.rs	66.66%	1 Missing :warning:

:loudspeaker: Thoughts on this report? Let us know!