run_assistant fails to run on a different node

[tamama]

Hey there,

I have run_assistant working perfectly fine in a docker container on one node, say node-18.

However, when I start the same docker container on a separate node, say node-10, it gets stuck. Here are the logs:

+May I know how to fix this? I would like the run_assistant service to be deployed in Kubernetes cluster.

Kind regards, Tamama

[root@f4987055771c assistant-sdk-cpp]# LD_LIBRARY_PATH=runtime bin/run_assistant --text_input "Hallo" --credentials_file asset/credentials.json --verbose
Using locale en-US
assistant_sdk robots_pem:
assistant_sdk CreateCustomChannel(embeddedassistant.googleapis.com:443, creds, arg)

[Fleker]

I'm not too familiar with how Kubernetes handles things. Is each node a separate process, running the sample individually?

How long do you wait for the program to run before determining it is unsuccessful?

[tamama]

I have compiled everything into a docker image, including credentials.json, on one machine (say node-18). Running run_assistant in a docker container works perfectly fine on node-18.

Then, I pull that docker image to another machine (say node-10), and start a docker container. It gets stuck (see logs below).

The docker image simply guarantees that both processes (node-18 and node-10) run exactly in the same clean environment. The only difference is that one runs on node-18, and another runs on node-10.

** For your second question, I wait for 5 minutes to determine that it is unsuccessful. Here is the proof.

[root@2370c6f65619 assistant-sdk-cpp]# date Thu May 3 02:25:42 UTC 2018 [root@2370c6f65619 assistant-sdk-cpp]# LD_LIBRARY_PATH=runtime bin/run_assistant --text_input "What time is it?" --credentials_file asset/credentials.json --verbose Using locale en-US assistant_sdk robots_pem: assistant_sdk CreateCustomChannel(embeddedassistant.googleapis.com:443, creds, arg)

^C [root@2370c6f65619 assistant-sdk-cpp]# date Thu May 3 02:30:45 UTC 2018 [root@2370c6f65619 assistant-sdk-cpp]#

[Fleker]

I really don't know why there'd be any issue. If the environments are the same, then there shouldn't be any software issues. Are you rebuilding the binary in each docker image, or using the same binary? Are there any specific hardware differences in each container that may cause a binary to run in one but not the other?

[tamama]

1. It is the same binary
2. It runs on the same OS (centos/7 base image)
3. I use the same cloud infrastructure

I still don't know why at this moment....

[tamama]

Maybe let me explain why I need to run it on different nodes.

1. Load balancing / redundancy : I want the assistant-SDK service to be a) highly available, and b) capable of handling huge network traffic.
2. The user-journey : a) Users will consume the service via Google Chrome browser where the audio-stream will be captured in javascript and redirected to (java) spring websocket server. b) The audio-stream will then be reverse-proxied to your underlying assistant-sdk c++ wrapped in a docker container deployed via Google kubernetes. c) Your c++ service will then consume the google assistant sdk grpc service hosted by Google.

This is the experiment that I want to showcase to my management

[Fleker]

Maybe, if you can, rebuild the binary on each container separately and see if that resolves your issue.

[tamama]

I will try this, and come back to you with updates soon.

Many thanks by the way!