mars
mars copied to clipboard
[BUG] Ray executor fail on test_bincount_execution
Describe the bug A clear and concise description of what the bug is.
setup = <mars.deploy.oscar.session.SyncSession object at 0x1762c9520>
def test_bincount_execution(setup):
rs = np.random.RandomState(0)
raw = rs.randint(0, 9, (100,))
raw[raw == 3] = 0
raw_weights = rs.rand(100)
# test non-chunked
a = tensor(raw)
result = bincount(a).execute().fetch()
expected = np.bincount(raw)
np.testing.assert_array_equal(result, expected)
weights = tensor(raw_weights)
result = bincount(a, weights=weights).execute().fetch()
expected = np.bincount(raw, weights=raw_weights)
np.testing.assert_array_equal(result, expected)
# test chunked
a = tensor(raw, chunk_size=13)
> result = bincount(a, chunk_size_limit=5).execute().fetch()
mars/tensor/statistics/tests/test_statistics_execution.py:528:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
mars/core/entity/tileables.py:462: in execute
result = self.data.execute(session=session, **kw)
mars/core/entity/executable.py:144: in execute
return execute(self, session=session, **kw)
mars/deploy/oscar/session.py:1888: in execute
return session.execute(
mars/deploy/oscar/session.py:1682: in execute
execution_info: ExecutionInfo = fut.result(
../../.pyenv/versions/3.8.13/lib/python3.8/concurrent/futures/_base.py:444: in result
return self.__get_result()
../../.pyenv/versions/3.8.13/lib/python3.8/concurrent/futures/_base.py:389: in __get_result
raise self._exception
mars/deploy/oscar/session.py:1868: in _execute
await execution_info
../../.pyenv/versions/3.8.13/lib/python3.8/asyncio/tasks.py:695: in _wrap_awaitable
return (yield from awaitable.__await__())
mars/deploy/oscar/session.py:105: in wait
return await self._aio_task
mars/deploy/oscar/session.py:953: in _run_in_background
raise task_result.error.with_traceback(task_result.traceback)
mars/services/task/supervisor/processor.py:372: in run
await self._process_stage_chunk_graph(*stage_args)
mars/services/task/supervisor/processor.py:250: in _process_stage_chunk_graph
chunk_to_result = await self._executor.execute_subtask_graph(
mars/services/task/execution/ray/executor.py:551: in execute_subtask_graph
meta_list = await asyncio.gather(*output_meta_object_refs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
awaitable = ObjectRef(1e360ffa862f8fe3ffffffffffffffffffffffff0100000001000000)
@types.coroutine
def _wrap_awaitable(awaitable):
"""Helper for asyncio.ensure_future().
Wraps awaitable (an object with __await__) into a coroutine
that will later be wrapped in a Task by ensure_future().
"""
> return (yield from awaitable.__await__())
E ray.exceptions.RayTaskError(TypeError): ray::execute_subtask() (pid=76452, ip=127.0.0.1)
E File "/Users/po.lb/Work/mars/mars/services/task/execution/ray/executor.py", line 185, in execute_subtask
E execute(context, chunk.op)
E File "/Users/po.lb/Work/mars/mars/core/operand/core.py", line 491, in execute
E result = executor(results, op)
E File "/Users/po.lb/Work/mars/mars/tensor/statistics/bincount.py", line 214, in execute
E op._execute_reduce(ctx, op)
E File "/Users/po.lb/Work/mars/mars/tensor/statistics/bincount.py", line 193, in _execute_reduce
E for input_key in op.inputs[0].op.source_keys:
E TypeError: 'NoneType' object is not iterable
../../.pyenv/versions/3.8.13/lib/python3.8/asyncio/tasks.py:695: RayTaskError(TypeError)

It seems that there are some bugs in bin count operand when shuffle by FETCH_BY_INDEX mode.
To Reproduce To help us reproducing this bug, please provide information below:
- Your Python version
- The version of Mars you use
- Versions of crucial packages, such as numpy, scipy and pandas
- Full stack of the error.
- Minimized code to reproduce the error.
Expected behavior A clear and concise description of what you expected to happen.
Additional context Add any other context about the problem here.