ray icon indicating copy to clipboard operation
ray copied to clipboard
verified publisher icon ray-project

[Core] ray.ActorID.nil().job_id

Open minerharry opened this issue 6 months ago • 1 comments

What happened + What you expected to happen

Accessing the job_id field of a nil ActorID (just the getattribute, not even executing the method) results in a full python crash instead of raising an appropriate exception.

Versions / Dependencies

Tested on ray 3.0.0.dev0 and 2.47.0, python 3.9 and 3.10

Reproduction script

Repro code:

from ray import ActorID
ActorId.nil().job_id

Output: [sometimes the stacktrace just says "Unknown", the following is from a self-built install)

[2025-06-16 22:29:27,301 C 92497 92497] id.cc:160:  Check failed: !IsNil() 
*** StackTrace Information ***
/home/miner/github/ray/python/ray/_raylet.so(+0x1612377) [0x76e597612377] ray::operator<<()
/home/miner/github/ray/python/ray/_raylet.so(_ZN3ray6RayLogD1Ev+0x514) [0x76e597616124] ray::RayLog::~RayLog()
/home/miner/github/ray/python/ray/_raylet.so(_ZNK3ray7ActorID5JobIdEv+0x10f) [0x76e59745888f] ray::ActorID::JobId()
/home/miner/github/ray/python/ray/_raylet.so(+0x859dbb) [0x76e596859dbb] __pyx_getprop_3ray_7_raylet_7ActorID_job_id()
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyObject_GenericGetAttrWithDict+0x2d6) [0x5ffe508c5cd6] _PyObject_GenericGetAttrWithDict
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x733) [0x5ffe508b7763] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1251e5) [0x5ffe508b61e5] _PyEval_EvalCode
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalCodeWithName+0x48) [0x5ffe508b5e98] _PyEval_EvalCodeWithName
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(PyEval_EvalCodeEx+0x39) [0x5ffe508b5e49] PyEval_EvalCodeEx
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(PyEval_EvalCode+0x1b) [0x5ffe5096074b] PyEval_EvalCode
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1d4324) [0x5ffe50965324] builtin_exec
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x137163) [0x5ffe508c8163] cfunction_vectorcall_FASTCALL
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x3af) [0x5ffe508b73df] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1d6cd2) [0x5ffe50967cd2] gen_send_ex
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x68e4) [0x5ffe508bd914] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1d6cd2) [0x5ffe50967cd2] gen_send_ex
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x68e4) [0x5ffe508bd914] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1d6cd2) [0x5ffe50967cd2] gen_send_ex
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1421ac) [0x5ffe508d31ac] method_vectorcall_O
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x664) [0x5ffe508b7694] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1369fa) [0x5ffe508c79fa] function_code_fastcall
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x3af) [0x5ffe508b73df] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1369fa) [0x5ffe508c79fa] function_code_fastcall
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x664) [0x5ffe508b7694] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1251e5) [0x5ffe508b61e5] _PyEval_EvalCode
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x143b6b) [0x5ffe508d4b6b] method_vectorcall
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x1167) [0x5ffe508b8197] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1369fa) [0x5ffe508c79fa] function_code_fastcall
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x664) [0x5ffe508b7694] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1369fa) [0x5ffe508c79fa] function_code_fastcall
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x664) [0x5ffe508b7694] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1369fa) [0x5ffe508c79fa] function_code_fastcall
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x664) [0x5ffe508b7694] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1251e5) [0x5ffe508b61e5] _PyEval_EvalCode
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x143b6b) [0x5ffe508d4b6b] method_vectorcall
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(PyObject_Call+0xbc) [0x5ffe508d52ec] PyObject_Call
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x369d) [0x5ffe508ba6cd] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1251e5) [0x5ffe508b61e5] _PyEval_EvalCode
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyFunction_Vectorcall+0xd9) [0x5ffe508c7759] _PyFunction_Vectorcall
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x3af) [0x5ffe508b73df] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1251e5) [0x5ffe508b61e5] _PyEval_EvalCode
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalCodeWithName+0x48) [0x5ffe508b5e98] _PyEval_EvalCodeWithName
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(PyEval_EvalCodeEx+0x39) [0x5ffe508b5e49] PyEval_EvalCodeEx
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(PyEval_EvalCode+0x1b) [0x5ffe5096074b] PyEval_EvalCode
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1fcc2a) [0x5ffe5098dc2a] run_eval_code_obj
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1f90f3) [0x5ffe5098a0f3] run_mod
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x942c4) [0x5ffe508252c4] pyrun_file.cold
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(PyRun_SimpleFileExFlags+0x1c0) [0x5ffe50984020] PyRun_SimpleFileExFlags
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(Py_RunMain+0x344) [0x5ffe509815e4] Py_RunMain
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(Py_BytesMain+0x37) [0x5ffe50954577] Py_BytesMain
/lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca) [0x76e59f82a1ca]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b) [0x76e59f82a28b] __libc_start_main
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1c348e) [0x5ffe5095448e]

Issue Severity

None

minerharry avatar Jun 17 '25 02:06 minerharry

Interestingly, this also happens for TaskID.nil().job_id(), but only when you execute the method - unlike ActorID.nil().job_id, simply accessing TaskID.nil().job_id doesn't crash. It also doesn't crash on TaskID.nil().actor_id or actor_id() - those just return a nil actorid. perhaps a nil TaskID / ActorID should also return nil JobIDs?

Stack trace for TaskID.nil().job_id():

[2025-06-16 23:09:58,727 C 95161 95161] id.cc:160:  Check failed: !IsNil() 
*** StackTrace Information ***
/home/miner/github/ray/python/ray/_raylet.so(+0x1612377) [0x787217812377] ray::operator<<()
/home/miner/github/ray/python/ray/_raylet.so(_ZN3ray6RayLogD1Ev+0x514) [0x787217816124] ray::RayLog::~RayLog()
/home/miner/github/ray/python/ray/_raylet.so(_ZNK3ray7ActorID5JobIdEv+0x10f) [0x78721765888f] ray::ActorID::JobId()
/home/miner/github/ray/python/ray/_raylet.so(_ZNK3ray6TaskID5JobIdEv+0x33) [0x787217658933] ray::TaskID::JobId()
/home/miner/github/ray/python/ray/_raylet.so(+0x86e600) [0x787216a6e600] __pyx_pw_3ray_7_raylet_6TaskID_15job_id()
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x664) [0x5b0914579694] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1251e5) [0x5b09145781e5] _PyEval_EvalCode
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalCodeWithName+0x48) [0x5b0914577e98] _PyEval_EvalCodeWithName
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(PyEval_EvalCodeEx+0x39) [0x5b0914577e49] PyEval_EvalCodeEx
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(PyEval_EvalCode+0x1b) [0x5b091462274b] PyEval_EvalCode
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1d4324) [0x5b0914627324] builtin_exec
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x137163) [0x5b091458a163] cfunction_vectorcall_FASTCALL
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x3af) [0x5b09145793df] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1d6cd2) [0x5b0914629cd2] gen_send_ex
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x68e4) [0x5b091457f914] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1d6cd2) [0x5b0914629cd2] gen_send_ex
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x68e4) [0x5b091457f914] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1d6cd2) [0x5b0914629cd2] gen_send_ex
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1421ac) [0x5b09145951ac] method_vectorcall_O
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x664) [0x5b0914579694] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1369fa) [0x5b09145899fa] function_code_fastcall
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x3af) [0x5b09145793df] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1369fa) [0x5b09145899fa] function_code_fastcall
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x664) [0x5b0914579694] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1251e5) [0x5b09145781e5] _PyEval_EvalCode
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x143b6b) [0x5b0914596b6b] method_vectorcall
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x1167) [0x5b091457a197] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1369fa) [0x5b09145899fa] function_code_fastcall
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x664) [0x5b0914579694] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1369fa) [0x5b09145899fa] function_code_fastcall
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x664) [0x5b0914579694] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1369fa) [0x5b09145899fa] function_code_fastcall
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x664) [0x5b0914579694] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1251e5) [0x5b09145781e5] _PyEval_EvalCode
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x143b6b) [0x5b0914596b6b] method_vectorcall
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(PyObject_Call+0xbc) [0x5b09145972ec] PyObject_Call
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x369d) [0x5b091457c6cd] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1251e5) [0x5b09145781e5] _PyEval_EvalCode
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyFunction_Vectorcall+0xd9) [0x5b0914589759] _PyFunction_Vectorcall
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalFrameDefault+0x3af) [0x5b09145793df] _PyEval_EvalFrameDefault
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1251e5) [0x5b09145781e5] _PyEval_EvalCode
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(_PyEval_EvalCodeWithName+0x48) [0x5b0914577e98] _PyEval_EvalCodeWithName
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(PyEval_EvalCodeEx+0x39) [0x5b0914577e49] PyEval_EvalCodeEx
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(PyEval_EvalCode+0x1b) [0x5b091462274b] PyEval_EvalCode
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1fcc2a) [0x5b091464fc2a] run_eval_code_obj
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1f90f3) [0x5b091464c0f3] run_mod
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x942c4) [0x5b09144e72c4] pyrun_file.cold
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(PyRun_SimpleFileExFlags+0x1c0) [0x5b0914646020] PyRun_SimpleFileExFlags
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(Py_RunMain+0x344) [0x5b09146435e4] Py_RunMain
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(Py_BytesMain+0x37) [0x5b0914616577] Py_BytesMain
/lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca) [0x78722142a1ca]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b) [0x78722142a28b] __libc_start_main
/home/miner/miniforge3/envs/ray_wsl_develop/bin/python3.9(+0x1c348e) [0x5b091461648e]

minerharry avatar Jun 17 '25 03:06 minerharry

Hey, interested in what your use case is for this? Can bump the priority if you have a way to get nil actor ids. I agree ideally we shouldn't crash, if you want to push a fix yourself, we can review it.

dayshah avatar Jun 17 '25 21:06 dayshah

I personally don't have a use case, but I'm working on type-stubbing the ray.includes files and noticed that those fields show up in ActorID. nil ActorIDs are used sporadically around the python codebase, I'll look to see if any are critical. As to the fix, I'm woefully inexperienced in Cython (and especially Cppython) but there may be an easy fix.

minerharry avatar Jun 17 '25 22:06 minerharry