Mingming Liu
Mingming Liu
Hi, Does the model use op `SimpleMLCreateModelResource`? Could you check out https://www.tensorflow.org/tfx/serving/custom_op and link the custom op with binary?
tensorflow_model_server with --model_base_path pointing to a gs:// bucket getting too long to start.
To investigate model loading time within model server binary, metrics in https://github.com/tensorflow/tensorflow/blob/23ddb02643654685f839e17a2ce72916775e5fe5/tensorflow/cc/saved_model/loader.cc#L43 would give insight. To further investigate the latency between docker and gs://, folks in docker forum or google...
tensorflow_model_server with --model_base_path pointing to a gs:// bucket getting too long to start.
IIUC both {gs, docker} and {s3, kubernetes} is about TFS running in a container so similar setups from this perspective. I think the advice of further breaking down e2e latency...
tensorflow_model_server with --model_base_path pointing to a gs:// bucket getting too long to start.
To confirm about the question, TFS model server docker image could load local models pretty fast, but it takes long to read from cloud file storage (assuming the saved model...
> Hi Roman, The motivating use case is allow benchmark users to inject custom implementations of profilers to measure as accurately as possible (e.g., no need to measure SetUp or...
> Then this doesn't do the right thing still - how does this deal with `PauseTiming()`? {Pause,Resume}Timing and profilers are added to solve relatively orthogonal problems. - {Pause,Resume}Timing are exposed....
> My point is that it may be fine for your use-case to ignore `PauseTiming()`, > but it clearly won't be true for everyone's use-case. The current way of injection...