Pycapnp Calculator example in Docker Not Working - Exception: Message is too deeply-nested or contains cycles

[nathaliechang]

I am working on pycapnp client and server applications in Docker, and tried to dockerize the calculator example. When creating containers for both the calculator_client.py and calculator_server.py and launching them on the same host using docker-compose, I obtain the following error from the client:

File "capnp/lib/capnp.pyx", line 1946, in capnp.lib.capnp._RemotePromise.wait capnp.lib.capnp.KjException: (remote):0: disconnected: remote exception: expected nestingLimit > 0; Message is too deeply-nested or contains cycles. See capnp::ReaderOptions. stack: 7f9c419460 7f9c40dc60 7f9c3b29bb 7f9c3a6eeb 7f9c3a6ffb 7f9c32169b 7f9cc4e47f 7f9cc8be63 7f9cc26d8f 7f9cc883af 7f9cc870b3 7f9cc86de3 7f9cc86d4f 7f9cd2182b 7f9cd39c1b 7f9cd39b67 7f9cbe06b7 7f9cbe03d7 7f9cd44837 7f9cd44413 7f9c9f7217 557e4a58b7

NOTE: When performing the same exercise of dockerizing the capnp C++ calculator example and launching with docker-compose, communication is successful with no exceptions obtained.

Further testing done: running the calculator_server Docker container only, and sending the calculator client requests by running the calculator_client.py script directly, ie.

• In one terminal: docker run --rm -it -p 9876:9876 calcserver_orig:0

• In another terminal: python calculator_client.py 172.17.0.2:9876

Exception is thrown: Evaluating a literal... Traceback (most recent call last): File "calculator_client.py", line 302, in <module> main(parse_args().host) File "calculator_client.py", line 72, in main response = read_promise.wait() File "capnp/lib/capnp.pyx", line 2122, in capnp.lib.capnp._RemotePromise.wait capnp.lib.capnp.KjException: (remote):0: disconnected: remote exception: expected nestingLimit > 0; Message is too deeply-nested or contains cycles. See capnp::ReaderOptions. stack: 7fb14d1e58 7fb14d0888 7fb1465113 7fb1454237 7fb14543b7 7fb13df743 4ae91b

Any help would be greatly appreciated!

Dockerfile.server: ARG BASE_IMAGE=python FROM $BASE_IMAGE

WORKDIR /root RUN apt-get update && apt-get install -y automake autogen build-essential make cmake gcc g++ libssl-dev RUN pip install --user pkgconfig==1.5.1 Cython==0.29.21 RUN pip install --user pycapnp==1.0.0 RUN mkdir test COPY calculator_server.py /root/test/ COPY calculator.capnp /root/test/

EXPOSE 9876

CMD ["sh", "-c", "/root/test/calculator_server.py *:9876"]

Dockerfile.client: ARG BASE_IMAGE=python FROM $BASE_IMAGE

WORKDIR /root RUN apt-get update && apt-get install -y automake autogen build-essential make cmake gcc g++ libssl-dev RUN pip install --user pkgconfig==1.5.1 Cython==0.29.21 RUN pip install --user pycapnp==1.0.0 RUN mkdir test COPY calculator_client.py /root/test/ COPY calculator.capnp /root/test/

EXPOSE 9876

CMD ["python", "/root/test/calculator_client.py", "server:9876"]

docker-compose.yaml services: server: image: calcserver_orig:0 tty: true # docker run -t ports: - "9876:9876" client: image: calcclient_orig:0 tty: true # docker run -t depends_on: [server]

[haata]

Does the same thing happen with async_calculator or async_ssl_calculator?

The default example uses socket files created in python and passed to the capnproto c++ library. While I'm not entirely certain why you're seeing this (likely something related to docker and how python is creating the socket is adding something extra that the capnproto c++ library doesn't like). This was the main reason I had to move to asyncio to get SSL to work with pycapnp.

async_calculator is compatible with calculator examples (both client and server).

[nathaliechang]

Thanks very much for the quick response! I tried dockerizing the async_calculator_server and client and communication is successful. However, when running the async_calculator, timings are much slower compared to the calculator example (both measured when not in docker). Is this expected from using asyncio? Running in docker and timing are important concerns, and am wondering how to improve timing of the async_calculator. Fyi am using unix sockets atm.

Thanks in advance for any insight.

[haata]

Yeah, right now there's a polling loop inside pycapnp that needs to be fixed: https://github.com/capnproto/pycapnp/issues/256. I haven't had a lot of time to look at this recently, but I'm happy to look at patches.

Not all of these are relevant to you, but are areas where performance will be reduced.

[master]: rg sleep                                                                                                                                                                                                                                                           
test/test_examples.py
68:            time.sleep(0.1)
104:                time.sleep(0.1)

test/test_capability.py
301:        lambda x: time.sleep(.1)
303:        lambda x: time.sleep(.1)

examples/test.sh
8:sleep 0.1
14:sleep 0.5

examples/async_ssl_server.py
90:            await asyncio.sleep(0.01)

examples/tester.bash
5:	sleep 1
7:	sleep 0.5
9:	sleep 1

examples/calculator_server.py
137:        time.sleep(0.001)

examples/thread_server.py
39:        time.sleep(0.001)

examples/async_calculator_server.py
70:            await asyncio.sleep(0.01)

examples/async_ssl_calculator_server.py
74:            await asyncio.sleep(0.01)

examples/async_reconnecting_ssl_client.py
60:            await asyncio.sleep(1)
145:            time.sleep(1)

examples/async_server.py
83:            await asyncio.sleep(0.01)

docs/quickstart.rst
606:                await asyncio.sleep(0.01)

capnp/lib/capnp.pyx
2138:            await asyncio.sleep(0.01)
2505:            await asyncio.sleep(0.01)
2622:            await asyncio.sleep(0.01)
2684:            await asyncio.sleep(0.01)

[Wiskerke]

I was trying out the synchronous calculator example code on windows 10 and got the same exception. Experimenting with it, I found out a couple of things:

The problem does not always occur, but it seems to trigger randomly when starting the server. After the server is started, all client runs would either succeed or they all would fail. It seems to be about 50% chance that it occurs for me.

In the server I replaced the:

   while True:
        server.poll_once()
        time.sleep(0.001)

with:

server.run_forever()

And with that change I no longer could reproduce the exception. The only side effect of this change is pressing 'CTRL-C' on the command line no longer works. So, there might be something going wrong with poll_once...

[edevil]

I am also seeing the same issue. My tests run fine on a amd64 linux, the same version of packages fail in arm64 with: Failed: remote exception: expected nestingLimit > 0; Message is too deeply-nested or contains cycles. See capnp::ReaderOptions.

I am already using server.run_forever().

I am also using unix sockets, but apparently I can't pass an already connected one to TwoPartyServer, despite what the documentation claims. It fails in https://github.com/capnproto/pycapnp/blob/master/capnp/lib/capnp.pyx#L2690 due to port_promise not having been initialized.

I am trying to avoid using Python asyncio, because the UX is bad, but it seems like the only way forward here?