Iman Tabrizian

I have not seen this issue before. Do you have access to the logs when the Pod restarts?

@khaykingleb Thanks for your feature request. I think this is a reasonable request. @krishung5 Do you have any thoughts regarding this request?

Thanks for reporting this issue. I have filed a ticket for the team to investigate.

@eeeeeunjung > and when i use BLS, i can only cancel the outer request, i can not cancel the sub request sent in the bls's execute function Good point! I...

Thanks for reporting this issue. I have filed a bug for this issue.

I think this is a reasonable ask. @GuanLuo / @rmccorm4 any thoughts? My only concern is that this might be a breaking change if the user relies on these logs...

@Jack47 I think this is a reasonable feature request. Is the main goal of this library to only help with debugging/development of Python backend models? I think once the development...

This has been requested in another context too. Unfortunately, this is not possible right now. This enhancement makes sense to me but I don't think we will get to it...

Hi @philipp-schmidt, yes, this is an important feature and we're working on it. The main challenge is that the schedulers/rate limiter currently do not take into account where the model...

@philipp-schmidt Could you please share the model repository and the system which you conducted this experiment on? I think creating separate models should provide the perf improvement that you're looking...