Ruiyang Wang

[data] ray_tqdm does not work with numba

1

### What happened + What you expected to happen `numba` is a JIT lib to compile python code to accelerate. It tries to type all functions and if it can't...

[POC][core] GcsClient async binding, aka remove PythonGcsClient.

1

This PR is a proof of concept that, we *can* do bindings for Async C++ APIs. Specifically, we can wrap a C++ `ray::gcs::GcsClient` Async APIs (by callbacks) into Python Async...

Creates a direct Cython binding for `ray::gcs::GcsClient` and replaces the existing `PythonGcsClient` binding. The new binding is enabled by default; one can switch back with `RAY_USE_OLD_GCS_CLIENT=1`. The new binding is...

[core] Add convenience method `RAY_LOG(INFO).WithField(job_id)`.

2

We added structured logging `RAY_LOG(INFO).WithField(kLogKeyJobID, job_id)`. However this is still a bit cumbersome and error prone, since one can write kLogKeyJobID with a NodeID and so on. This PR introduces...

[core] Add 1s timeout in RPC to CoreWorkerService.NumPendingTasks in GcsJobManager::HandleGetAllJobInfo

1

Critical Dashboard API `GET /api/jobs` sends RPC to `JobInfoGcsService.GetAllJobInfo`, where the GCS sends RPC to each to `CoreWorkerService.NumPendingTasks` for the info of "how many running tasks do ya have rn?"....

### What happened + What you expected to happen When you have a task returning a Ray Object, and the task is in "waiting dependencies" status; and you do `ray.cancel(ref)`...

[core] Add from_binary and from_hex to all IDs.

2

Some Ray IDs have those fields and some don't. Unify them with a list of methods and test. Note ObjectID is not in the list.

Like actor_head.py, we now update DataSource.nodes on delta. It first queries all node infos, then subscribes node deltas. Each delta updates: 1. DataSource.nodes[node_id] 2. DataSource.agents[node_id] 3. a warning generated after...

When starting a worker for a task, we have: - early bind: runtime env, is_gpu, is_actor - late bind: job_id, root_detached_id Now, move the latter 3 fields from "early bind"...

We already have this filter in GCS for state_manager.py. Also exposes it to node_head.py. Smoke tested each arg manually.