serving

serving copied to clipboard

Refresh aspired servables/versions following config update

6

Currently when the configured model list is updated via a call to `handleReloadConfigRequest`, the request thread blocks until any newly added models become available. Their availability however depends on the...

njhill

Sort input/output in PreProcessPrediction

4

In direct_session.cc https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/common_runtime/direct_session.cc#L1514, it always emplaces key to executors_, then a lot of keys are added to map, which leads to a lot of memory usage. If 10 input tensors,...

zhjunqin

addresses this https://github.com/tensorflow/serving/issues/795 I think it's a feature the community really wants

yupbank

OP_REQUIRES failed at xla_ops : UNIMPLEMENTED: Could not find compiler for platform CUDA: NOT_FOUND

9

## Bug Report Does Tensorflow Serving support XLA compiled SavedModels ? or am I doing something wrong ? ### System information - **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**:...

kmkolasinski

Add health check to Dockerfile

4

# Feature Request Add a health check to the Dockerfile ## Describe the problem the feature is intended to solve I'm using a docker container on an edge device and...

alejones

java.lang.RuntimeException: Unexpected code Response{protocol=http/1.1, code=400, message=Bad Request, url=http://localhost:8501/v1/models/myfruit:predict}

4

**System information** operating system： `win11 64bit` tensorflow/serving version: `2.14.1 ` TFserving Deployment mode: `RESTful API` python requirements.txt： `tensorflow-cpu == 2.3.0 pyqt5 pillow opencv-python matplotlib` **Describe the problem** Hello everyone, my...

x13872140520

Why TF Serving using one CUDA Compute Stream

1

Trying to understand why TF uses one CUDA compute stream? Is there a metric which shows if ops are waiting to be scheduled on that one compute stream? I want...

ndeep27

## Feature Request May I know the expected release date of TensorFlow Serving with TensorFlow Runtime? For the background, recently, I found that tensorflow runtime is integrated into tensorflow/serving codes....

jeongukjae

OP_REQUIRES failed at xla_compile_on_demand_op.cc:290 : UNIMPLEMENTED: Could not find compiler for platform CUDA: NOT_FOUND

5

## Bug Report If this is a bug report, please fill out the following form in full: ### System information - **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: -...

rb-23

CUDA Graphs support for Tensorflow Serving

2

Does TF Serving support CUDA graphs?

ndeep27