Very Stuck

[raymondtri]

Hello Mt,

First of all, thank you for taking the time to put this together.

I am relatively new to TensorFlow, and I have been adapting this model to work with my data.

I am having some difficulty troubleshooting a problem, and I was wondering if you might be able to provide some guidance.

I can have trained and exported the model, and I am running tensorflow_model_server (the pre-compiled version for ubuntu, I can't get bazel build to work properly)

Currently, the script reaches line 51 (stub.Predict.future(request, 5.0)) and then hangs indefinitely. The timeout isn't triggered, nothing like that, it just sits.

Do you have any idea as to what could be causing this or where I could begin?

Thanks, Ray

[MtDersvan]

Hi @raymondtri ! Sorry for the late response( Can you provide a step-by-step command walkthrough of what you are doing? And any logs would be beneficial. I'll try to help you if it's something in my immediate domain of knowledge or related to the tutorial.

[raymondtri]

I have created a minimum reproducible test case here: https://github.com/raymondtri/client-test

The issue seems to be relating to GRPC's threading functions and Apache's mod_wsgi. There is something happening where mod_wsgi drops the processes which hold the stubs waiting for a response, and the responses end up getting orphaned.

[MtDersvan]

@raymondtri I see what you are saying. I've been able to reproduce your pure tf serving + flask (without mod_wsgi), and as you mention it is working properly. And it works well without mod_wsgi. So it is indeed a GRPC with Apache's mod_wsgi connection problem. I have never used Apache mod_wsgi before so I do not know what would be the best workaround, but I suggest you to open an issue in https://github.com/grpc/grpc or https://github.com/tensorflow/serving/ repository as folks there might be of better competence.