Sean Sheng

@withsmilo we are in the design phase of a model monitoring solution where we offer APIs for logging features and inference results and configurations for shipping the logs to a...

@andrewsi-z we have released a stable BentoML framework API in 1.0. You are welcome to contribute SnapML as one of the natively supported frameworks.

Should we implement max latency similar to the `deadline` feature in gRPC or have a 10 max latency PER runner?

@udevnl Thanks for your help with benchmarking BentoML performance. If neither CPU nor GPU is saturated, the bottleneck could be in the API or Runner code itself. Here are common...

There were CI test failures. Could you please take a look, @bojiang?

@fwindolf How's the progress on this? Any help we can provide?

@fwindolf, thanks for the update. No you are not blocking anyone. Just that we'd like to extend our help if needed. Looking forward to your PR.

An option is to add a `configuration` section in bentofile.yml.

- Configuration should be versioned - Configuration should be mapped to environment variable names

Thanks for compiling this PR, @aarnphm. On CLI, should we use the `--grpc` option or `serve-grpc` sub-command? Using `--grpc` makes sense if most of the other options are also applicable...