AbortionError(code=StatusCode.FAILED_PRECONDITION, details="Attempting to use uninitialized value conv2d/kernel [[Node: conv2d/kernel/read = Identity[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"](conv2d/kernel)]] [[Node: ArgMax/_3 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_94_ArgMax", tensor_type=DT_INT64, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]")

[ps-account]

Hi, I built the model in TF-serving-gpu, copied it to the server, and it starts fine.

When calling I get an error however:

root@a0bf24294b1b:/tf-serving-example/tensorflow-serving-example# python python/grpc_mnist_client.py --image ./data/0.png --model mnist --host localhost [[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 44. 140. 254. 183. 12. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 125. 229. 253. 253. 253. 65. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 68. 164. 217. 254. 253. 253. 253. 227. 164. 78. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 64. 246. 253. 253. 254. 253. 182. 253. 253. 253. 182. 78. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 72. 240. 253. 253. 253. 254. 68. 21. 178. 245. 253. 253. 243. 52. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 72. 232. 253. 253. 248. 223. 74. 13. 0. 0. 119. 253. 253. 253. 77. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 224. 253. 253. 253. 128. 0. 0. 0. 0. 0. 36. 214. 253. 253. 232. 56. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 6. 161. 251. 253. 253. 116. 4. 0. 0. 0. 0. 0. 0. 38. 253. 253. 253. 104. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 130. 253. 253. 248. 128. 4. 0. 0. 0. 0. 0. 0. 0. 7. 162. 253. 253. 104. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 36. 217. 253. 253. 223. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 134. 253. 253. 192. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 211. 254. 254. 223. 44. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 135. 255. 254. 255. 0. 0. 0. 0.] [ 0. 0. 0. 0. 87. 246. 253. 253. 55. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 134. 253. 253. 253. 0. 0. 0. 0.] [ 0. 0. 0. 0. 114. 253. 253. 246. 27. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 134. 253. 253. 253. 0. 0. 0. 0.] [ 0. 0. 0. 0. 254. 253. 253. 133. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 86. 191. 253. 253. 182. 0. 0. 0. 0.] [ 0. 0. 0. 0. 254. 253. 240. 16. 0. 0. 0. 0. 0. 0. 0. 0. 5. 75. 120. 245. 253. 253. 240. 73. 0. 0. 0. 0.] [ 0. 0. 0. 0. 148. 253. 249. 109. 22. 0. 0. 0. 0. 18. 31. 154. 184. 253. 253. 253. 253. 253. 190. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 105. 253. 253. 253. 218. 134. 134. 134. 134. 205. 254. 253. 253. 253. 253. 253. 253. 126. 28. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 6. 212. 253. 253. 253. 253. 253. 253. 253. 253. 255. 253. 253. 253. 253. 242. 84. 1. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 49. 158. 253. 253. 253. 253. 253. 253. 253. 236. 208. 199. 129. 164. 52. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 7. 104. 236. 253. 253. 164. 226. 104. 62. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]] Traceback (most recent call last): File "python/grpc_mnist_client.py", line 56, in run(args.host, args.port, args.image, args.model, args.signature_name) File "python/grpc_mnist_client.py", line 34, in run result = stub.Predict(request, 10.0) File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 309, in call self._request_serializer, self._response_deserializer) File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 195, in _blocking_unary_unary raise _abortion_error(rpc_error_call) grpc.framework.interfaces.face.face.AbortionError: AbortionError(code=StatusCode.FAILED_PRECONDITION, details="Attempting to use uninitialized value conv2d/kernel [[Node: conv2d/kernel/read = IdentityT=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]] [[Node: ArgMax/_3 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_94_ArgMax", tensor_type=DT_INT64, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]")

[ps-account]

I also found a discussion on that error here: https://stackoverflow.com/questions/45139423/tensorflow-error-failedpeconditionerror-attempting-to-use-uninitialized-variab

[ps-account]

( It could be related that I'm building this in TF-serving-gpu 1.9.0 )

[ps-account]

it probably is https://github.com/tensorflow/serving/issues/594

will try again later either moving your example to 1.9.0 or moving my tf-serving-gpu to 1.8

[ps-account]

Hmmm, I did a pip install tensorflow-gpu==1.9.0 and run tf-serving-gpu 1.9.0 but I got the same error