mars icon indicating copy to clipboard operation
mars copied to clipboard

[BUG] Ray task backend no progress

Open chaokunyang opened this issue 2 years ago • 2 comments

Describe the bug Ray task mode doesn't update progress until whole task finished: image To Reproduce To help us reproducing this bug, please provide information below:

  1. Your Python version: 3.7.9
  2. The version of Mars you use: master
  3. Versions of crucial packages, such as numpy, scipy and pandas
  4. Full stack of the error.
  5. Minimized code to reproduce the error.
def test_groupby(n=10):
    from datetime import datetime
    start = datetime.now()
    df = md.DataFrame(
    mt.random.rand(n * 500, 4, chunk_size=500),
    columns=list('abcd'))
    # print(df.sum().execute())
    result = df.groupby(['a']).apply(lambda pdf: pdf).execute()
    duration = datetime.now() - start
    return result, duration
mars.new_session(n_worker=10, n_cpu=10*2, backend="ray")
test_groupby(200)

Expected behavior A clear and concise description of what you expected to happen.

Additional context Add any other context about the problem here.

chaokunyang avatar Aug 09 '22 11:08 chaokunyang

I'll fix it soon.

zhongchun avatar Aug 09 '22 11:08 zhongchun

There is a status bar and an a continuously updated value as follows: image And i add some log it will print: image

zhongchun avatar Aug 11 '22 03:08 zhongchun