BentoML

BentoML copied to clipboard

Is your feature request related to a problem? Please describe.

Currently there is not too much visibility for the user to visualize or to understand how BentoML's micro-batching works. having more logs around how BentoML makes a decision and what is happening in the batching layer should help with that.

cc @bojiang

Describe the solution you'd like

Describe alternatives you've considered

Additional context

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

In debug mode, RunnerApp should:

• log the batch size for each run
• log the latency for each run