[ray 2.20.0] perf regressions

[can-anyscale]

REGRESSION 9.79%: client__get_calls (THROUGHPUT) regresses from 1091.5724282284737 to 984.7185148948039 (9.79%) in 2.20.0/microbenchmark.json
REGRESSION 7.83%: single_client_get_object_containing_10k_refs (THROUGHPUT) regresses from 13.704970913400441 to 12.632060871624994 (7.83%) in 2.20.0/microbenchmark.json
REGRESSION 7.82%: client__put_calls (THROUGHPUT) regresses from 835.8219066517684 to 770.4735069321608 (7.82%) in 2.20.0/microbenchmark.json
REGRESSION 6.44%: single_client_tasks_and_get_batch (THROUGHPUT) regresses from 8.314592004219296 to 7.778792239134901 (6.44%) in 2.20.0/microbenchmark.json
REGRESSION 6.19%: n_n_actor_calls_with_arg_async (THROUGHPUT) regresses from 2846.036342831333 to 2669.9811846586795 (6.19%) in 2.20.0/microbenchmark.json
REGRESSION 4.92%: placement_group_create/removal (THROUGHPUT) regresses from 823.8896629199388 to 783.3481516266015 (4.92%) in 2.20.0/microbenchmark.json
REGRESSION 4.74%: single_client_put_gigabytes (THROUGHPUT) regresses from 19.613041636801512 to 18.683362618370772 (4.74%) in 2.20.0/microbenchmark.json
REGRESSION 4.69%: tasks_per_second (THROUGHPUT) regresses from 610.9508780114498 to 582.3046176755182 (4.69%) in 2.20.0/benchmarks/many_tasks.json
REGRESSION 4.16%: client__1_1_actor_calls_sync (THROUGHPUT) regresses from 523.0280053260831 to 501.2526439748014 (4.16%) in 2.20.0/microbenchmark.json
REGRESSION 3.98%: multi_client_put_gigabytes (THROUGHPUT) regresses from 39.026311549309554 to 37.47472636769277 (3.98%) in 2.20.0/microbenchmark.json
REGRESSION 3.40%: n_n_actor_calls_async (THROUGHPUT) regresses from 28165.509673837572 to 27206.931303151337 (3.40%) in 2.20.0/microbenchmark.json
REGRESSION 2.97%: 1_1_async_actor_calls_sync (THROUGHPUT) regresses from 1382.8140547973987 to 1341.7887372798139 (2.97%) in 2.20.0/microbenchmark.json
REGRESSION 2.72%: 1_n_actor_calls_async (THROUGHPUT) regresses from 8797.319487481269 to 8558.218903786727 (2.72%) in 2.20.0/microbenchmark.json
REGRESSION 2.58%: actors_per_second (THROUGHPUT) regresses from 653.272142233422 to 636.4202043276684 (2.58%) in 2.20.0/benchmarks/many_actors.json
REGRESSION 2.24%: client__tasks_and_get_batch (THROUGHPUT) regresses from 0.9074349140473464 to 0.8871100522697384 (2.24%) in 2.20.0/microbenchmark.json
REGRESSION 2.23%: single_client_tasks_sync (THROUGHPUT) regresses from 987.9154572339097 to 965.8386012454489 (2.23%) in 2.20.0/microbenchmark.json
REGRESSION 2.23%: single_client_tasks_async (THROUGHPUT) regresses from 8175.996977132117 to 7993.559119696471 (2.23%) in 2.20.0/microbenchmark.json
REGRESSION 2.06%: 1_n_async_actor_calls_async (THROUGHPUT) regresses from 7864.108971520267 to 7701.7490843523765 (2.06%) in 2.20.0/microbenchmark.json
REGRESSION 1.70%: 1_1_actor_calls_async (THROUGHPUT) regresses from 8900.061966026988 to 8748.782799582932 (1.70%) in 2.20.0/microbenchmark.json
REGRESSION 1.55%: single_client_put_calls_Plasma_Store (THROUGHPUT) regresses from 5222.55884804985 to 5141.775101569989 (1.55%) in 2.20.0/microbenchmark.json
REGRESSION 1.39%: pgs_per_second (THROUGHPUT) regresses from 23.471191969494587 to 23.144902235160945 (1.39%) in 2.20.0/benchmarks/many_pgs.json
REGRESSION 1.39%: single_client_wait_1k_refs (THROUGHPUT) regresses from 5.425389917194363 to 5.3499932380060775 (1.39%) in 2.20.0/microbenchmark.json
REGRESSION 0.59%: 1_1_async_actor_calls_with_args_async (THROUGHPUT) regresses from 2411.2175991602116 to 2396.8963139443463 (0.59%) in 2.20.0/microbenchmark.json
REGRESSION 351.66%: dashboard_p95_latency_ms (LATENCY) regresses from 3552.838 to 16046.576 (351.66%) in 2.20.0/benchmarks/many_tasks.json
REGRESSION 333.00%: dashboard_p99_latency_ms (LATENCY) regresses from 126.705 to 548.637 (333.00%) in 2.20.0/benchmarks/many_pgs.json
REGRESSION 321.30%: dashboard_p99_latency_ms (LATENCY) regresses from 4635.591 to 19529.559 (321.30%) in 2.20.0/benchmarks/many_tasks.json
REGRESSION 115.75%: dashboard_p95_latency_ms (LATENCY) regresses from 19.066 to 41.134 (115.75%) in 2.20.0/benchmarks/many_pgs.json
REGRESSION 42.54%: dashboard_p99_latency_ms (LATENCY) regresses from 3787.161 to 5398.368 (42.54%) in 2.20.0/benchmarks/many_actors.json
REGRESSION 21.21%: dashboard_p95_latency_ms (LATENCY) regresses from 2575.396 to 3121.637 (21.21%) in 2.20.0/benchmarks/many_actors.json
REGRESSION 6.19%: dashboard_p50_latency_ms (LATENCY) regresses from 4.02 to 4.269 (6.19%) in 2.20.0/benchmarks/many_nodes.json
REGRESSION 6.05%: stage_0_time (LATENCY) regresses from 10.757715463638306 to 11.408677577972412 (6.05%) in 2.20.0/stress_tests/stress_test_many_tasks.json
REGRESSION 4.79%: 10000_get_time (LATENCY) regresses from 22.577659229999995 to 23.659871710999994 (4.79%) in 2.20.0/scalability/single_node.json
REGRESSION 4.74%: dashboard_p99_latency_ms (LATENCY) regresses from 206.312 to 216.1 (4.74%) in 2.20.0/benchmarks/many_nodes.json
REGRESSION 4.11%: 107374182400_large_object_time (LATENCY) regresses from 29.17789060800004 to 30.37595573499999 (4.11%) in 2.20.0/scalability/single_node.json
REGRESSION 3.30%: 3000_returns_time (LATENCY) regresses from 5.479904671000014 to 5.660801429999992 (3.30%) in 2.20.0/scalability/single_node.json
REGRESSION 3.21%: time_to_broadcast_1073741824_bytes_to_50_nodes (LATENCY) regresses from 16.808720801999982 to 17.34891581100004 (3.21%) in 2.20.0/scalability/object_store.json
REGRESSION 1.72%: avg_pg_create_time_ms (LATENCY) regresses from 0.8821673768768409 to 0.897374594595741 (1.72%) in 2.20.0/stress_tests/stress_test_placement_group.json