Support for websockets testing

[mehadi92]

Is python websockets testing available?

[timbmg]

If your docker container supports it, it should work. Do you have a concrete example?

[mehadi92]

@timbmg Thank you for your response,

I have a websocket server same as asr_server.py

And I have a docker file base on this script

FROM alphacep/kaldi-vosk-server:latest

ENV MODEL_VERSION 0.1

RUN python3 -m pip install requests

RUN mkdir /opt/vosk-model-en \
   && cd /opt/vosk-model-en
COPY models /opt/vosk-model-bng/model
RUN ls -la /opt/vosk-model-en/model/graph
WORKDIR /opt/vosk-server/websocket
COPY src .
RUN ls -la /opt/vosk-server/websocket
CMD [ "python3", "./asr_server.py", "/opt/vosk-model-en/model" ]

Now I would like to test this Dockerfile image. How can I do it using testcontainers-python

Thanks

[timbmg]

You can start any container for testing with

from testcontainers.core.container import DockerContainer

with DockerContainer(image="my-image"):
    # ...

Just build the image before and tag it, then run your test code.

You might also want to integrate the build in your test. You could (1) write a makefile where you first build and then run the tests (2) use a fixture to build the image using the python docker client

[menon92]

@timbmg I'm getting this error ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 2700) when running the bellow code. Is there anything I'm messing with? Thanks

from testcontainers.core.container import DockerContainer

with DockerContainer(image="my-image") as dc:
    async def hello():
        async with websockets.connect("ws://127.0.0.1:2700") as websocket:
            await websocket.send("Hello world!")
            await websocket.recv()

    asyncio.run(hello())

[timbmg]

Which container are you running? I.e. with what did you replace "my-image"?

[menon92]

@timbmg Build a docker image using [Dockerfile.kaldi-en-in](https://github.com/alphacep/vosk-server/blob/master/docker/Dockerfile.kaldi-en-in file and replace "my-image" with my local docker image name

[menon92]

@timbmg I also try with DockerCompose() and it has the same issue. You can regenerate the issue as follows,

Project structure

simpleWS/
├── docker-compose.yml
├── Dockerfile
├── server.py
└── test_server.py

1. docker-compose.yml file contains

version: '2'

services:
    simple_ws:
        build: ./
        container_name: simple_ws
        image: simple_ws
        ports:
            - "2700:2700"
        network_mode: host

2. Dockerfile contains

FROM python:3.8-slim-buster

RUN python3 -m pip install websockets

WORKDIR /app

COPY ./ /app

EXPOSE 2700
CMD [ "python3", "server.py"]

3. server.py contains

#!/usr/bin/env python

import asyncio
import websockets

async def echo(websocket):
    async for message in websocket:
        await websocket.send(message)

async def main():
    async with websockets.serve(echo, "127.0.0.1", 2700):
        await asyncio.Future()  # run forever

asyncio.run(main())

4. test_server.py contains

import asyncio
import websockets
from testcontainers.compose import DockerCompose
from testcontainers.core.container import DockerContainer

with DockerCompose('./', compose_file_name='docker-compose.yml', build=True):
    async def hello():
        async with websockets.connect("ws://127.0.0.1:2700") as websocket:
            await websocket.send("Hello world!")
            print(await websocket.recv())

    asyncio.run(hello())

But when I run python test_server.py it gives me the following error

$ python test_server.py                                                                                                                                  (vosk-api) 
/usr/lib/python3/dist-packages/paramiko/transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
Building simple_ws
Step 1/6 : FROM python:3.8-slim-buster
 ---> ea4599aa52dc
Step 2/6 : RUN python3 -m pip install websockets
 ---> Using cache
 ---> 9af32a52612f
Step 3/6 : WORKDIR /app
 ---> Using cache
 ---> 1ba52233d46e
Step 4/6 : COPY ./ /app
 ---> Using cache
 ---> ed1c081c146d
Step 5/6 : EXPOSE 2700
 ---> Using cache
 ---> 307f2cc3b0ea
Step 6/6 : CMD [ "python3", "server.py"]
 ---> Using cache
 ---> df8338937aa2
Successfully built df8338937aa2
Successfully tagged simple_ws:latest
Creating simple_ws ... done
/usr/lib/python3/dist-packages/paramiko/transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
Stopping simple_ws ... done
Removing simple_ws ... done
Traceback (most recent call last):
  File "test_server.py", line 12, in <module>
    asyncio.run(hello())
  File "/home/menon/anaconda3/envs/vosk-api/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/menon/anaconda3/envs/vosk-api/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "test_server.py", line 8, in hello
    async with websockets.connect("ws://127.0.0.1:2700") as websocket:
  File "/home/menon/anaconda3/envs/vosk-api/lib/python3.8/site-packages/websockets/legacy/client.py", line 633, in __aenter__
    return await self
  File "/home/menon/anaconda3/envs/vosk-api/lib/python3.8/site-packages/websockets/legacy/client.py", line 650, in __await_impl_timeout__
    return await asyncio.wait_for(self.__await_impl__(), self.open_timeout)
  File "/home/menon/anaconda3/envs/vosk-api/lib/python3.8/asyncio/tasks.py", line 494, in wait_for
    return fut.result()
  File "/home/menon/anaconda3/envs/vosk-api/lib/python3.8/site-packages/websockets/legacy/client.py", line 654, in __await_impl__
    transport, protocol = await self._create_connection()
  File "/home/menon/anaconda3/envs/vosk-api/lib/python3.8/asyncio/base_events.py", line 1025, in create_connection
    raise exceptions[0]
  File "/home/menon/anaconda3/envs/vosk-api/lib/python3.8/asyncio/base_events.py", line 1010, in create_connection
    sock = await self._connect_sock(
  File "/home/menon/anaconda3/envs/vosk-api/lib/python3.8/asyncio/base_events.py", line 924, in _connect_sock
    await self.sock_connect(sock, address)
  File "/home/menon/anaconda3/envs/vosk-api/lib/python3.8/asyncio/selector_events.py", line 496, in sock_connect
    return await fut
  File "/home/menon/anaconda3/envs/vosk-api/lib/python3.8/asyncio/selector_events.py", line 528, in _sock_connect_cb
    raise OSError(err, f'Connect call failed {address}')
ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 2700)

[timbmg]

You also need to expose the port when running the image; EXPOSE by itself does not do it.

here's a MWE:

exposed_port = 80
with DockerContainer(image="containous/whoami").with_exposed_ports(exposed_port) as container:
    print(f"Container port {exposed_port} is exposed on {container.get_exposed_port(exposed_port)} on host.")
    while True: continue

This maps a random, available port to port 80 of the container.

According to the docker docs on EXPOSE

The EXPOSE instruction does not actually publish the port. It functions as a type of documentation between the person who builds the image and the person who runs the container, about which ports are intended to be published. To actually publish the port when running the container, use the -p flag on docker run to publish and map one or more ports, or the -P flag to publish all exposed ports and map them to high-order ports.

[menon92]

@timbmg Still facing the issue, does it support async? Can't test the below code under context manager

async def hello():
        async with websockets.connect("ws://127.0.0.1:2700") as websocket:
            await websocket.send("Hello world!")
            print(await websocket.recv())

asyncio.run(hello())

[timbmg]

Does it work with the docker cli? Ie when you start your container with docker run and then run the python code that connects.

Is there maybe some waiting required before the server becomes available?

[menon92]

@timbmg I use this docker image https://hub.docker.com/repository/docker/menon92/simple_ws

and run this docker container using,

docker run -it --net host menon92/simple_ws

Now I can run the bellow code without any issue

async def hello():
    async with websockets.connect(f"ws://127.0.0.1:2700") as websocket:
        await websocket.send("Hello world!")
        print(await websocket.recv())

result = asyncio.run(hello())

[menon92]

@timbmg Adding explicit delay solves the issue, Here is an example that solves this issue. Is there any other way to address this problem?

with DockerContainer(image="menon92/simple_ws", network_mode='host') as container:
        async def hello():
            async with websockets.connect("ws://127.0.0.1:2700") as websocket:
                await websocket.send("Hello world!")
                print(await websocket.recv())

        import time
        time.sleep(10)
        result = asyncio.run(hello())
        print("Complete` ...")

[timbmg]

There is this wait_container_is_ready function that might be helpful for you. It's used as a decorator in child classes, but you could also implement a single function. See for example here.

[tillahoffmann]

I'm closing this one for now, as @timbmg's comment should address the question of explicit waits. Feel free to follow up if there are further issues.