serving issues

Record runtime latency with signature name

I wrote the patch that can achieve what I want in #1959 . Close #1959

Docker Container for Support between tf-io and tf-serving.

11

Using the tf.io ops in the tf.serving ecosystem would be a large development convenience and likely decrease inference latency. Can there be an official docker build or documentation to integrate...

Ouwen

type:feature

needs prio

custom-ops

stale

Add arguments to configure gRPC completion queue parameters

2

I discovered a performance issue that Tensorflow Serving has an unexplainable and significant network delay for tail latencies when facing higher loads of traffic. My setup was a client and...

yarikmarkov

type:feature

stat:awaiting tensorflower

Tensorflow Serving fails to serve tflite model with multiple signatures

6

## Bug Report ### System information - **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Ubuntu 20.04 / Kubernetes on AWS - **TensorFlow Serving installed from (source or binary)**: binary...

flesaint

stat:awaiting tensorflower

type:bug

ShardedReadPtrs can overflow shards_ array in unusual situations

3

## Bug Report If this is a bug report, please fill out the following form in full: ### System information - **OS Platform and Distribution: Linux Ubuntu 16.04 - **TensorFlow...

arai-fortanix

stat:contributions welcome

stat:awaiting tensorflower

type:bug

Add support for half type for the http restful api

4

Half type is widely used in the deeplearning inference, but the tf-serving doesn't support half type in the restful api, I submit a pr to solve this problem, please check.

venuswu

cla: yes

capping resources assigned to each model in multi model serving

3

Is there a way to cap the number (e.g. CPU cores, CUDA MPS threads) of resources assigned to each model in a multi-model tensorflow server? The only way (straightforward way...

saeid93

type:feature

stat:awaiting tensorflower

Tensorflow Serving should handle SIGTERM correctly

5

TF Serving should terminate gracefully when SIGTERM is received. This is especially important for docker / kubernetes use cases when a process is terminated gracefully or is killed have very...

miguelvr