xianxian.zhang comments

Repositories
Issues
Comments

Results 4 comments of


                                            xianxian.zhang

fix: memory issue when push large bentos

> What are the bugs within the current implementation? context in https://bentoml-team.slack.com/archives/C02QLC8RB5W/p1695088745009929 as a summary, currently it will take too much memory when do a `bentoml push` since it use...

fix: memory issue when push large bentos

> Looks like unit tests are failing, maybe because of the requests change...? > > Should we just use a `SpooledTemporaryFile` for everything? - checked the ut failed logs, seems...

bug: Concurrent requests with the streaming feature produce parallel calls to the runner

Definitely runner can run in parallel especially for the VLLM case we can do batching to enhance performance. We can not make such assumption in BentoML. BTW, If you want...

bug: Working Service for HTTP gives error with Grpc "type doesn't support gRPC serving"

Hi. thanks for reporting. Bentoml 1.2 Service have not impl Grpc support yet.