mars icon indicating copy to clipboard operation
mars copied to clipboard

Published 20 hours ago • verified publisher icon mars-project

Optimize transferring small objects for storage

Open hekaisheng opened this issue 2 years ago • 4 comments

What do these changes do?

Related issue number

This PR implements a method that send small objects directly to receiver without split them into small blocks.

Fixes #xxxx

Check code requirements

  • [ ] tests added / passed (if needed)
  • [ ] Ensure all linting tests pass, see here for how to run them

hekaisheng avatar May 20 '22 09:05 hekaisheng

Seems vineyard cannot pass the ut for this PR, @sighingnow do you have to time to help to see what's going on?

qinxuye avatar Jun 09 '22 05:06 qinxuye

Seems vineyard cannot pass the ut for this PR, @sighingnow do you have to time to help to see what's going on?

I would take a look later today.

sighingnow avatar Jun 09 '22 06:06 sighingnow

Seems vineyard cannot pass the ut for this PR, @sighingnow do you have to time to help to see what's going on?

I would take a look later today.

Sorry for the delay... Will be fixed today.

sighingnow avatar Jun 14 '22 01:06 sighingnow

I failed to reproduce the failure if only run the single test case. But I found the test case test_execute_describe failed both for default storage and vineyard storage under this PR.

I have tried to debugging but failed to address the root cause. I think it is not a vineyard-only issue, @hekaisheng could you please help to try the test_execute_describe in your environment?

You could update the params to test the default storage only. The failre could be reproduced by

pytest -s -vvv mars/deploy/oscar/tests/test_local.py -k test_execute_describe

sighingnow avatar Jun 15 '22 01:06 sighingnow