xoscar
xoscar copied to clipboard
Support fury as an optional serializer
What do these changes do?
Related issue number
import xorbits
import time
from xorbits.remote import spawn
from xoscar.serialization.core import FurySerializer
from dataclasses import dataclass
from typing import List
@dataclass
class Foo:
f1: int
f2: int
f3: float
f4: int
f5: int
f6: List[int]
FurySerializer.register(Foo)
import logging
logging.basicConfig(level=logging.DEBUG)
if __name__ == "__main__":
xorbits.init(config={"third_party_modules":["t1"]})
def f1(x):
return x
foos = [Foo(f1=5000, f2=2 ** 32, f3=1 / 3, f4=100, f5=200, f6=[11, 1234]) for i in range(500)]
start = time.time()
results = []
for _ in range(100):
r1 = spawn(f1, foos)
results.append(r1)
xorbits.run(results)
print(f"cost: {time.time() - start}")
This test code may be crash when fury deserializing, but if it does not crash, the performance is better than Mars default serializers.
Use fury(if not crash): 9.764425039291382 Mars default: 13.590802669525146
Fixes #xxxx
Check code requirements
- [ ] tests added / passed (if needed)
- [ ] Ensure all linting tests pass
Codecov Report
Attention: 1 lines
in your changes are missing coverage. Please review.
Comparison is base (
2a041ae
) 89.12% compared to head (d43d8f4
) 88.98%.
Files | Patch % | Lines |
---|---|---|
python/xoscar/serialization/pyfury.py | 83.33% | 1 Missing :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## main #78 +/- ##
==========================================
- Coverage 89.12% 88.98% -0.15%
==========================================
Files 48 48
Lines 4010 4012 +2
Branches 764 764
==========================================
- Hits 3574 3570 -4
- Misses 351 356 +5
- Partials 85 86 +1
Flag | Coverage Δ | |
---|---|---|
unittests | 88.80% <83.33%> (-0.15%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Is there any things I can help for this PR?
Is there any things I can help for this PR?
fury Python is hard to use, could you help fix the crash issue?