yqchen

Petals doesn't deal with server failure properly

4

Hi there, we'd like to report our findings on testing Petals' availability of fault tolerance. We note that the current implementation of the method _step_ in the class __ServerInferenceSession_ from...

batch processing/parallel processing

1

Hi there, does Petals currenly support batch processing/parallel processing? For example, to increase resource usage or system throughput, we would like to see servers parallelly processing multiple prompts at the...

Performance improving chances in the future

1

Hi there, I've been following this work for a few months and found it's really an amazing idea to run LLMs over the Internet, while I'm also trying to improve...