Feature request: secure WebSocket (wss)

[beyonlo]

Hello!

Congratulations for the great project.

I would like to know if you have intention to support secure WebSocket (use SSL over WebSocket) on the Microdot.

Thank you.

[miguelgrinberg]

SSL and WebSocket are completely independent features. I have the intention to investigate adding SSL support. I wasn't considering websocket, but it's not out of the question either. My concern is that with each addition the size of the package will continue to grow and the lower-end microprocessors will not be able to run anymore, so I'll have to think about doing this in a way that doesn't affect the smaller platforms.

[beyonlo]

SSL and WebSocket are completely independent features. I have the intention to investigate adding SSL support. I wasn't considering websocket, but it's not out of the question either. My concern is that with each addition the size of the package will continue to grow and the lower-end microprocessors will not be able to run anymore, so I'll have to think about doing this in a way that doesn't affect the smaller platforms.

Well, the microdot webserver/framework do not support HTTPS (SSL over HTTP) right? I think, even lower-end microcontrollers (like as ESP32-C3) would be great a secure connection over browser to configure/operate. So, if you add SSL on HTTP, the websocket can be the same way - I think that SSL is what will use more memory, right? Well, already exists a official module uwebsocket used by REPL. Here has a official uwebsocket module example (https://github.com/micropython/micropython/blob/master/extmod/webrepl/webrepl.py) but unfortunately do not use the uasyncio, but maybe that can help to no increase the microdot so much (using already exists uwebsocket module instead create your own). And already exists a HTTP Server official example running with SSL https://github.com/micropython/micropython/blob/master/examples/network/http_server_ssl.py but that not works with uasyncio too, unfortunately.

Well, my intention (my application) is to put in ESP32 works a secure HTTPS Server (to serve pages) + secure WebSocket Server, both running on uasyncio. I posted in MicroPython page, a time ago, a issue looking for a Simple example of uasyncio WebSocket Server (secure - with SSL) https://github.com/micropython/micropython/issues/8177

I think that most work to support ssl + asyncio / websockets was done, in many PRs, like this PR shows https://github.com/micropython/micropython/pull/5611

Thank you in advance!

[miguelgrinberg]

I believe uwebsocket does not work with uasyncio. But in general I agree, I'm open to add support for both SSL and WS.

[beyonlo]

I believe uwebsocket does not work with uasyncio. But in general I agree, I'm open to add support for both SSL and WS.

Hey @miguelgrinberg

That's will be great :)

I see that you have two microdot on /src/: microdot.py for people that want to use with thread microdot_asyncio.py to works with uasyncio/asyncio

Well, just as suggestion, maybe a option is have more options, so user can to choose what want: microdot_https.py Secure WebServer
microdot_https_asyncio.py Secure WebServer uasyncio/asyncio microdot_https_wss.py Secure WebServer + secure WebSocket Server microdot_https_wss_asyncio.py Secure WebServer + secure WebSocket Server uasyncio/asyncio

Particularly I liked so much uasyncio/asyncio :)

Going forward, with this feature in microdot is possible great applications. I can have the mobile app, desktop app and web app (microdot serving) all working with ESP32 using the same protocol, the websocket. That is great not just because is the same protocol, but is bidirectional protocol. Actually, using just http request on webserver, I need to refresh every 1-5s the page to have new data from my sensors. With websockets that polling is not more necessary. That was just a example, but I think that secure WebServer + secure WebSocket Server + uasyncio/asyncio will be an amazing feature to provide great applications/products.

Thank you so much!

[miguelgrinberg]

Update: WebSocket support is coming in the next release of Microdot. Will investigate SSL support after that.

[beyonlo]

Update: WebSocket support is coming in the next release of Microdot. Will investigate SSL support after that.

@miguelgrinberg That is a amazing news!!! I'm very interested!!

Actually I started to use this project https://github.com/marcidy/micropython-uasyncio-webexample to have a HTTP Server and a Websocket Server. My intention is to have just one HTML page with an entire large web application using javaScript (Jquery), exactly as that project works. This way is good because all processing will be on Client side (browser), not in the Microcontroller. As the my application has no access to the internet, is needed to put inside HTTP Server the JQuery lib do use on that unique page, exactly as that project works as well. Look here https://github.com/marcidy/micropython-uasyncio-webexample/tree/main/www

Will be possible to works in that way (just one page.html and put JQuery lib inside HTTP Server) on the Microdot as well?

Thank you very much.

[miguelgrinberg]

Will be possible to works in that way (just one page.html and put JQuery lib inside HTTP Server) on the Microdot as well?

You can already serve static files with Microdot. I have added an example of how to do this just a few days ago: https://github.com/miguelgrinberg/microdot/tree/main/examples/static.

And for websocket it will be just a normal route. This isn't finished code, but if you want to have a look, here is a WebSocket echo example: https://github.com/miguelgrinberg/microdot/blob/websocket/examples/websocket/echo_async.py

[beyonlo]

You can already serve static files with Microdot. I have added an example of how to do this just a few days ago: https://github.com/miguelgrinberg/microdot/tree/main/examples/static.

Excellent! That will solve the use case to serve images, JQuery lib and others kind off static files! :)

And for websocket it will be just a normal route. This isn't finished code, but if you want to have a look, here is a WebSocket echo example: https://github.com/miguelgrinberg/microdot/blob/websocket/examples/websocket/echo_async.py

Great, very clean with uasyncio! Congrats!

On the next Microdot release, do you have plans do provide together to the WebSocket Server feature a HTML page (index.html) with an WebSocket Client example running on the browser? Like as a start point (ready to go)!

As soon the next version is released I will start to use Microdot! :)

[miguelgrinberg]

an WebSocket Client example running on the browser?

Here is one from another project of mine: https://github.com/miguelgrinberg/flask-sock/blob/main/examples/templates/index.html. I did not test it, but I think it should directly work with the echo example I linked above.

[beyonlo]

an WebSocket Client example running on the browser?

Here is one from another project of mine: https://github.com/miguelgrinberg/flask-sock/blob/main/examples/templates/index.html. I did not test it, but I think it should directly work with the echo example I linked above.

Very good. I will test that, thanks :)

Will investigate SSL support after that.

To support SSL on HTTP Server and WebSocket Server will be amazing. I don't know if you know, but recently was created a ticket (micropython/micropython#8915) that has a relation with this feature on Microdot - maybe help.

[beyonlo]

Hi @miguelgrinberg

I tested the WebSocket branch using this WebSocket client example (https://github.com/miguelgrinberg/flask-sock/blob/main/examples/templates/index.html) and works very well!

However after I finished the test, I observed an error on the terminal. That error do not stopped the send/receive data from the browser (WebSocket Client) test, and I can't to reproduce that error anymore. I will paste here this test with the error, that maybe you can investigate if is really an error or not.

websocket_async.py example (a merge from hello_async.py and echo_async.py):

from microdot_asyncio import Microdot
from microdot_asyncio_websocket import websocket

app = Microdot()

htmldoc = '''<!doctype html>
<html>
  <head>
    <title>Flask-Sock Demo</title>
  </head>
  <body>
    <h1>Flask-Sock Demo</h1>
    <p>Type <b>close</b> to end the connection.</p>
    <div id="log"></div>
    <br>
    <form id="form">
      <label for="text">Input: </label>
      <input type="text" id="text" autofocus>
    </form>
    <script>
      const log = (text, color) => {
        document.getElementById('log').innerHTML += `<span style="color: ${color}">${text}</span><br>`;
      };

      const socket = new WebSocket('ws://' + location.host + '/echo');
      socket.addEventListener('message', ev => {
        log('<<< ' + ev.data, 'blue');
      });
      socket.addEventListener('close', ev => {
        log('<<< closed');
      });
      document.getElementById('form').onsubmit = ev => {
        ev.preventDefault();
        const textField = document.getElementById('text');
        log('>>> ' + textField.value, 'red');
        socket.send(textField.value);
        textField.value = '';
      };
    </script>
  </body>
</html>
'''

@app.route('/echo')
@websocket
async def echo(request, ws):
    c = 0
    while True:
        data = await ws.receive()
        print('Received data from client: {} - type is: {}'.format(data, type(data)))
        data = '{}_{}'.format(data, c)
        await ws.send(data)
        c += 1


@app.route('/')
async def hello(request):
    return htmldoc, 200, {'Content-Type': 'text/html'}


@app.route('/shutdown')
async def shutdown(request):
    request.app.shutdown()
    return 'The server is shutting down...'


app.run(debug=True)

Output terminal:

$ mpremote run websocket_async.py 
Starting async server on 0.0.0.0:5000...
GET / 200
Received data from client: s - type is: <class 'str'>
Received data from client: a - type is: <class 'str'>
Received data from client: asdas - type is: <class 'str'>
Received data from client: asdasd - type is: <class 'str'>
Received data from client: close - type is: <class 'str'>
Received data from client: freeee - type is: <class 'str'>
Task exception wasn't retrieved
future: <Task> coro= <generator object 'serve' at 3fcafef0>
Traceback (most recent call last):
  File "uasyncio/core.py", line 1, in run_until_complete
  File "microdot_asyncio.py", line 261, in serve
  File "microdot_asyncio.py", line 321, in handle_request
  File "microdot_asyncio.py", line 154, in write
  File "uasyncio/stream.py", line 1, in stream_awrite
  File "uasyncio/stream.py", line 1, in drain
OSError: [Errno 104] ECONNRESET
Received data from client: asda - type is: <class 'str'>
Received data from client: asaaa - type is: <class 'str'>
Received data from client: a - type is: <class 'str'>
Received data from client: a - type is: <class 'str'>
Received data from client: a - type is: <class 'str'>
Received data from client: a - type is: <class 'str'>
Received data from client: aasdasd - type is: <class 'str'>
Received data from client: as - type is: <class 'str'>
Received data from client: das - type is: <class 'str'>
Received data from client: d - type is: <class 'str'>
Received data from client: asd - type is: <class 'str'>
Received data from client: as - type is: <class 'str'>
Received data from client: close - type is: <class 'str'>

Output Browser:

[beyonlo]

@miguelgrinberg I did tests with the example above (websocket_async.py) to check the compatibility with different browsers.

Works: Chrome, Chromium, Microsoft Edge and Opera. Do not works: Firefox (errors below).

I have not tested others browsers, just these above.

Terminal output for Firefox test:

$ mpremote run websocket_async.py 
Starting async server on 0.0.0.0:5000...
GET / 200
Traceback (most recent call last):
  File "microdot_asyncio.py", line 353, in dispatch_request
  File "microdot_asyncio.py", line 410, in _invoke_handler
  File "microdot_asyncio_websocket.py", line 93, in wrapper
  File "microdot_asyncio_websocket.py", line 69, in websocket_upgrade
  File "microdot_asyncio_websocket.py", line 7, in handshake
  File "microdot_websocket.py", line 50, in _handshake_response
NameError: name 'abort' isn't defined
GET /favicon.ico 404
GET /echo 500

Firefox browser output:

Thank you.

[miguelgrinberg]

Thanks, this is useful feedback.

[beyonlo]

Hello @miguelgrinberg

I did some more tests.

1. More browsers tested: Chrome for Android and Safari for Iphone (IOS). Both works.
2. I'm using ESP32-S3 with MicroPython 1.19.1, and all tests above (in others reply) was done via WiFi, using STA mode. Now I tested the ESP32-S3 as AP mode, and sometimes (not always) show this errors:

Ps1: one kind of error here is the same found in the reply above (that as running as STA mode). Others errors are a bit different. Ps2: one time was needed to stop the Microdot and start it again because client (browser) was not capable anymore to open the IP and Port of the AP (http://192.168.4.1:5000). I do not restarted the WiFi on the ESP32-S3 (AP mode), just restarted the Microdot and and all back to works. I understand that Microdot is transparent of network, but is something that can be done to fix that?

Received data from client: S - type is: <class 'str'>
Received data from client: D - type is: <class 'str'>
Received data from client: D - type is: <class 'str'>
Received data from client: D - type is: <class 'str'>
Traceback (most recent call last):
  File "microdot_asyncio.py", line 353, in dispatch_request
  File "microdot_asyncio.py", line 410, in _invoke_handler
  File "microdot_asyncio_websocket.py", line 95, in wrapper
  File "<stdin>", line 49, in echo
  File "microdot_asyncio_websocket.py", line 17, in receive
  File "microdot_asyncio_websocket.py", line 36, in _read_frame
  File "uasyncio/stream.py", line 1, in read
OSError: [Errno 104] ECONNRESET
Task exception wasn't retrieved
future: <Task> coro= <generator object 'serve' at 3fcc0400>
Traceback (most recent call last):
  File "uasyncio/core.py", line 1, in run_until_complete
  File "microdot_asyncio.py", line 261, in serve
  File "microdot_asyncio.py", line 321, in handle_request
  File "microdot_asyncio.py", line 139, in write
  File "uasyncio/stream.py", line 1, in stream_awrite
  File "uasyncio/stream.py", line 1, in drain
OSError: [Errno 104] ECONNRESET
GET / 200
GET / 200
GET /echo 200
GET /echo 200
Received data from client: Hhh - type is: <class 'str'>
Received data from client: JSON - type is: <class 'str'>
Traceback (most recent call last):
  File "microdot_asyncio.py", line 353, in dispatch_request
  File "microdot_asyncio.py", line 410, in _invoke_handler
  File "microdot_asyncio_websocket.py", line 95, in wrapper
  File "<stdin>", line 49, in echo
  File "microdot_asyncio_websocket.py", line 17, in receive
  File "microdot_asyncio_websocket.py", line 36, in _read_frame
  File "uasyncio/stream.py", line 1, in read
OSError: [Errno 104] ECONNRESET
Task exception wasn't retrieved
future: <Task> coro= <generator object 'serve' at 3fcae620>
Traceback (most recent call last):
  File "uasyncio/core.py", line 1, in run_until_complete
  File "microdot_asyncio.py", line 261, in serve
  File "microdot_asyncio.py", line 321, in handle_request
  File "microdot_asyncio.py", line 139, in write
  File "uasyncio/stream.py", line 1, in stream_awrite
  File "uasyncio/stream.py", line 1, in drain
OSError: [Errno 104] ECONNRESET
GET / 200
Received data from client: F - type is: <class 'str'>
Received data from client: G - type is: <class 'str'>
GET / 200
Received data from client: H - type is: <class 'str'>
Received data from client: JSON - type is: <class 'str'>
Traceback (most recent call last):
  File "microdot_asyncio.py", line 353, in dispatch_request
  File "microdot_asyncio.py", line 410, in _invoke_handler
  File "microdot_asyncio_websocket.py", line 95, in wrapper
  File "<stdin>", line 49, in echo
  File "microdot_asyncio_websocket.py", line 17, in receive
  File "microdot_asyncio_websocket.py", line 36, in _read_frame
  File "uasyncio/stream.py", line 1, in read
OSError: [Errno 104] ECONNRESET
Task exception wasn't retrieved
future: <Task> coro= <generator object 'serve' at 3fcbdbb0>
Traceback (most recent call last):
  File "uasyncio/core.py", line 1, in run_until_complete
  File "microdot_asyncio.py", line 261, in serve
  File "microdot_asyncio.py", line 321, in handle_request
  File "microdot_asyncio.py", line 139, in write
  File "uasyncio/stream.py", line 1, in stream_awrite
  File "uasyncio/stream.py", line 1, in drain
OSError: [Errno 104] ECONNRESET
Traceback (most recent call last):
  File "microdot_asyncio.py", line 315, in handle_request
  File "microdot_asyncio.py", line 69, in create
  File "microdot_asyncio.py", line 110, in _safe_readline
  File "uasyncio/stream.py", line 1, in readline
OSError: [Errno 104] ECONNRESET
Traceback (most recent call last):
  File "microdot_asyncio.py", line 353, in dispatch_request
  File "microdot_asyncio.py", line 410, in _invoke_handler
  File "microdot_asyncio_websocket.py", line 95, in wrapper
  File "<stdin>", line 49, in echo
  File "microdot_asyncio_websocket.py", line 17, in receive
  File "microdot_asyncio_websocket.py", line 36, in _read_frame
  File "uasyncio/stream.py", line 1, in read
OSError: [Errno 104] ECONNRESET
Task exception wasn't retrieved
future: <Task> coro= <generator object 'serve' at 3fcb4320>
Traceback (most recent call last):
  File "uasyncio/core.py", line 1, in run_until_complete
  File "microdot_asyncio.py", line 261, in serve
  File "microdot_asyncio.py", line 321, in handle_request
  File "microdot_asyncio.py", line 139, in write
  File "uasyncio/stream.py", line 1, in stream_awrite
  File "uasyncio/stream.py", line 1, in drain
OSError: [Errno 104] ECONNRESET
Task exception wasn't retrieved
future: <Task> coro= <generator object 'serve' at 3fcb68d0>
Traceback (most recent call last):
  File "uasyncio/core.py", line 1, in run_until_complete
  File "microdot_asyncio.py", line 261, in serve
  File "microdot_asyncio.py", line 321, in handle_request
  File "microdot_asyncio.py", line 139, in write
  File "uasyncio/stream.py", line 1, in stream_awrite
  File "uasyncio/stream.py", line 1, in drain
OSError: [Errno 104] ECONNRESET

[miguelgrinberg]

Looks like all the errors are caused by not handling the ECONNRESET error. I did not notice this error in my tests, it might be that they are used by the ES32 implementation, which I did not test. I'll look into handling this error properly.

[Carglglz]

Hi @beyonlo @miguelgrinberg As a part of a ongoing development effort in #8968 to bring SSLContext to MicroPython, I've done some test with microdot and so far the _thread version works (I've done tests in UNIX and ESP32 ports), using python requests (with verify=False since I'm using self-signed certs) and curl. Here is the diff to microdot.py:

git diff --staged src/*
diff --git a/src/microdot.py b/src/microdot.py
index 5b9e77e..48149a1 100644
--- a/src/microdot.py
+++ b/src/microdot.py
@@ -51,6 +51,8 @@ except ImportError:
     except ImportError:  # pragma: no cover
         socket = None

+import ssl
+

 def urldecode(string):
     string = string.replace('+', ' ')
@@ -847,7 +849,7 @@ class Microdot():
         """
         raise HTTPException(status_code, reason)

-    def run(self, host='0.0.0.0', port=5000, debug=False):
+    def run(self, host='0.0.0.0', port=5000, debug=False, key=None, cert=None):
         """Start the web server. This function does not normally return, as
         the server enters an endless listening loop. The :func:`shutdown`
         function provides a method for terminating the server gracefully.
@@ -882,7 +884,11 @@ class Microdot():
         self.server = socket.socket()
         ai = socket.getaddrinfo(host, port)
         addr = ai[0][-1]
-
+        ctx = None
+        if key:
+            ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
+            ctx.load_cert_chain(cert,
+                                keyfile=key)
         if self.debug:  # pragma: no cover
             print('Starting {mode} server on {host}:{port}...'.format(
                 mode=concurrency_mode, host=host, port=port))
@@ -893,12 +899,17 @@ class Microdot():
         while not self.shutdown_requested:
             try:
                 sock, addr = self.server.accept()
+                if key:
+                    ssl_sock = ctx.wrap_socket(sock, server_side=True)
             except OSError as exc:  # pragma: no cover
                 if exc.errno == errno.ECONNABORTED:
                     break
                 else:
                     raise
-            create_thread(self.handle_request, sock, addr)
+            if key:
+                create_thread(self.handle_request, ssl_sock, addr, ctx)
+            else:
+                create_thread(self.handle_request, sock, addr, ctx)

     def shutdown(self):
         """Request a server shutdown. The server will then exit its request
@@ -927,7 +938,7 @@ class Microdot():
                     f = 405
         return f

-    def handle_request(self, sock, addr):
+    def handle_request(self, sock, addr, ctx):
         if not hasattr(sock, 'readline'):  # pragma: no cover
             stream = sock.makefile("rwb")
         else:
@@ -955,6 +966,8 @@ class Microdot():
             print('{method} {path} {status_code}'.format(
                 method=req.method, path=req.path,
                 status_code=res.status_code))
+        if hasattr(ctx, 'reset'):
+            ctx.reset()

     def dispatch_request(self, req):
         if req:

And the hello_tls_context.py example I use for testing:

from microdot import Microdot

app = Microdot()

htmldoc = '''<!DOCTYPE html>
<html>
    <head>
        <title>Microdot Example Page</title>
    </head>
    <body>
        <div>
            <h1>Microdot Example Page</h1>
            <p>Hello from Microdot!</p>
            <p><a href="/shutdown">Click to shutdown the server</a></p>
        </div>
    </body>
</html>
'''


@app.route('/')
def hello(request):
    return htmldoc, 200, {'Content-Type': 'text/html'}


@app.route('/shutdown')
def shutdown(request):
    request.app.shutdown()
    return 'The server is shutting down...'

with open('ec-cakey.pem', 'rb') as keyb:
    key = keyb.read()

with open('ec-cacert.pem', 'rb') as certb:
    cert = certb.read()

app.run(debug=True, key=key, cert=cert)

Sadly I don't know when or even if my implementation of SSLContext will be merged.

I will try with microdot asyncio version too, although not sure when exactly will be that.

[miguelgrinberg]

@beyonlo I have put some fixes based on your reports:

• Firefox should work now
• the ECONNRESET error is (hopefully) handled properly
• an index.html file is now served in all the WebSocket examples when you connect to the root URL
• preliminary support for websocket connections in the test client (incomplete, more work is needed still)
• the websocket decorator was renamed to with_websocket for consistency

Let me know if things look better now.

[miguelgrinberg]

@Carglglz Thanks for doing this work! This is going to make my work a lot easier when I attempt to do this. Sounds like I might need to use the older (current) SSL implementation though, since it is unclear when or if yours is going to be merged, correct? I also want to see if it is possible to put the SSL support in a separate extension, as I'm doing with WebSocket and other features, so that it does not continue to make the main server larger.

[beyonlo]

Hello @miguelgrinberg

@beyonlo I have put some fixes based on your reports:

That's great!

• Firefox should work now

I confirm that it works!

• the ECONNRESET error is (hopefully) handled properly

Yes, that old errors do not show anymore, thank you. But now I did two other tests and show these errors:

1. Killing the Browser after connected.

Killing the Chrome on Android: Ps: the error happen every time that browser is killed.

Received data from client: D
Received data from client: D
Received data from client: S
Traceback (most recent call last):
  File "microdot_asyncio.py", line 353, in dispatch_request
  File "microdot_asyncio.py", line 410, in _invoke_handler
  File "microdot_asyncio_websocket.py", line 95, in wrapper
  File "<stdin>", line 17, in echo
  File "microdot_asyncio_websocket.py", line 17, in receive
  File "microdot_asyncio_websocket.py", line 37, in _read_frame
  File "microdot_websocket.py", line 63, in _parse_frame_header
IndexError: bytes index out of range
Task exception wasn't retrieved
future: <Task> coro= <generator object 'serve' at 3fccb1c0>
Traceback (most recent call last):
  File "uasyncio/core.py", line 1, in run_until_complete
  File "microdot_asyncio.py", line 261, in serve
  File "microdot_asyncio.py", line 321, in handle_request
  File "microdot_asyncio.py", line 145, in write
  File "uasyncio/stream.py", line 1, in stream_awrite
  File "uasyncio/stream.py", line 1, in drain
OSError: [Errno 104] ECONNRESET
Task exception wasn't retrieved
future: <Task> coro= <generator object 'serve' at 3fcca9a0>
Traceback (most recent call last):
  File "uasyncio/core.py", line 1, in run_until_complete
  File "microdot_asyncio.py", line 261, in serve
  File "microdot_asyncio.py", line 321, in handle_request
  File "microdot_asyncio.py", line 145, in write
  File "uasyncio/stream.py", line 1, in stream_awrite
  File "uasyncio/stream.py", line 1, in drain
OSError: [Errno 104] ECONNRESET
Received data from client: sfsd
Received data from client: f

Killing the Chromium on Ubuntu: Ps: the error happen just some times.

Received data from client: sd
Received data from client: a
Task exception wasn't retrieved
future: <Task> coro= <generator object 'serve' at 3fcc2450>
Traceback (most recent call last):
  File "uasyncio/core.py", line 1, in run_until_complete
  File "microdot_asyncio.py", line 261, in serve
  File "microdot_asyncio.py", line 321, in handle_request
  File "microdot_asyncio.py", line 145, in write
  File "uasyncio/stream.py", line 1, in stream_awrite
  File "uasyncio/stream.py", line 1, in drain
OSError: [Errno 104] ECONNRESET
Received data from client: ds
Received data from client: asdasd
Received data from client: a

Killing the Firefox on Ubuntu do not show errors

2. Happened a non-intentional lost network, and it back a few seconds after, and show this error: Ps: I tried to force network lost connection to reproduce the problem, but not success.

Received data from client: as
Received data from client: d
Received data from client: asd
Task exception wasn't retrieved
future: <Task> coro= <generator object 'serve' at 3fcc9f20>
Traceback (most recent call last):
  File "uasyncio/core.py", line 1, in run_until_complete
  File "microdot_asyncio.py", line 261, in serve
  File "microdot_asyncio.py", line 321, in handle_request
  File "microdot_asyncio.py", line 145, in write
  File "uasyncio/stream.py", line 1, in stream_awrite
  File "uasyncio/stream.py", line 1, in drain
OSError: [Errno 104] ECONNRESET
Received data from client: ada

• an index.html file is now served in all the WebSocket examples when you connect to the root URL

Excellent!

• preliminary support for websocket connections in the test client (incomplete, more work is needed still)

I never did/used unittests, but I will to study/check how it works and start to use the test_microdot_websocket.py as well in my tests. I think need just to instance the TestMicrodotWebSocket class and call the test_websocket_echo method, right? I checked that import unittest do not is builtin by default on MicroPython, but I see that there is the unittest for MicroPython.

• the websocket decorator was renamed to with_websocket for consistency

All right!

Additional comments:

1. I just noticed that this new version are not showing anymore IP:PORT that Microdot are running - show nothing. In the last version was showing something like as Running at 0.0.0.0:5000.
2. All tests above was done using the new echo_async.py on the ESP32-S3 running MicroPython 1.19.1. The ESP32-S3 was running as AP mode and my Notebook (Ubuntu) and my smartphone (Android) as STA mode, connected to ESP32-S3.

   ESP32-S3 - 192.168.4.1
   Notebook - 192.168.4.2
   Smartphone - 192.168.4.3

[miguelgrinberg]

@beyonlo I added fixes for the new errors. Thanks!

[raven703]

Confirm, that ECONNRESET has been fixed ! I`m using server side events in my ESP32 project and it was spamming in local each time browser reloads page forcing Microdot to stop serve pages. Now everything is fine, thank you :)

[beyonlo]

@miguelgrinberg I confirm that is working without ECONNRESET errors.

Thank you!

[beyonlo]

Hi @miguelgrinberg

Could you please to provide a simple example a bit different than echo_async.py using asyncio as well? I mean, where the WebSocket Server is capable to send data to the WebSocket Client without the WebSocket Client send a request (like as the echo example), and where the WebSocket Server is capable as well to select (choose) for what WebSocket Clients to send the data? Just a example as start point.

I would like to have many WebSockets Clients (from browsers and from Desktop/Mobile), simultaneously connected do the WebSocket Server and the clients will be stayed connected. So the WebSocket Server can receive/send data from/to all them, but for some critical data (like as alarms/events), for example, the WebSocket Server need to send the data just for some specific WebSocket Clients.

Thank you in advance!

[miguelgrinberg]

@beyonlo I don't have any examples, but all you need to do is store the ws objects for all your clients in some sort of data structure (maybe a dict), so that you can access them when you need to send something to one or more clients. The route can take care of receiving data from clients, but for sending you can send from anywhere, as long as you have access to the ws object.

[beyonlo]

@beyonlo I don't have any examples, but all you need to do is store the ws objects for all your clients in some sort of data structure (maybe a dict), so that you can access them when you need to send something to one or more clients. The route can take care of receiving data from clients, but for sending you can send from anywhere, as long as you have access to the ws object.

@miguelgrinberg with this explanation now I understand how to do it - I will to try, thanks!

In my tests I found more errors. I don't know when that errors happened and I was no capable to reproduce. Note that the second error (ENOTCONN) is different from all others errors reported before.

Received data: s
Send data: s -> 9
1070296480 1070296736
Received data: Vvvg
Send data: Vvvg -> 1
1070295920 1070296048
Traceback (most recent call last):
  File "microdot_asyncio.py", line 353, in dispatch_request
  File "microdot_asyncio.py", line 410, in _invoke_handler
  File "microdot_asyncio_websocket.py", line 101, in wrapper
  File "microdot_asyncio_websocket.py", line 33, in close
  File "microdot_asyncio_websocket.py", line 28, in send
  File "uasyncio/stream.py", line 1, in stream_awrite
  File "uasyncio/stream.py", line 1, in drain
OSError: [Errno 104] ECONNRESET
Task exception wasn't retrieved
future: <Task> coro= <generator object 'serve' at 3fcaed40>
Traceback (most recent call last):
  File "uasyncio/core.py", line 1, in run_until_complete
  File "microdot_asyncio.py", line 261, in serve
  File "microdot_asyncio.py", line 321, in handle_request
  File "microdot_asyncio.py", line 140, in write
  File "uasyncio/stream.py", line 1, in stream_awrite
  File "uasyncio/stream.py", line 1, in drain
OSError: [Errno 128] ENOTCONN


GET / 200
GET /static/jquery-ui.min.css 200
GET /static/jquery-3.6.0.min.js 200
GET /static/jquery-ui-1.13.2.min.js 200
---------------------
<Request object at 3fcb79e0> <WebSocket object at 3fcb7aa0>
<class 'Request'> <class 'WebSocket'>
1070299616 1070299808
---------------------
1070299616 1070299808
Received data: asdsa
Send data: asdsa -> 0
1070299616 1070299808
Received data: asd

Edit:

Happened one more time:

Received data: free
Send data: free -> 76
1070301872 1070302064
Traceback (most recent call last):
  File "microdot_asyncio.py", line 353, in dispatch_request
  File "microdot_asyncio.py", line 410, in _invoke_handler
  File "microdot_asyncio_websocket.py", line 101, in wrapper
  File "microdot_asyncio_websocket.py", line 33, in close
  File "microdot_asyncio_websocket.py", line 28, in send
  File "uasyncio/stream.py", line 1, in stream_awrite
  File "uasyncio/stream.py", line 1, in drain
OSError: [Errno 104] ECONNRESET
Task exception wasn't retrieved
future: <Task> coro= <generator object 'serve' at 3fcac1d0>
Traceback (most recent call last):
  File "uasyncio/core.py", line 1, in run_until_complete
  File "microdot_asyncio.py", line 261, in serve
  File "microdot_asyncio.py", line 321, in handle_request
  File "microdot_asyncio.py", line 140, in write
  File "uasyncio/stream.py", line 1, in stream_awrite
  File "uasyncio/stream.py", line 1, in drain
OSError: [Errno 128] ENOTCONN

[miguelgrinberg]

@beyonlo This is useful, thanks. I think I've fixed these new errors now.

[beyonlo]

@miguelgrinberg Errors do not happen anymore, thanks!

Do you know if is possible to limit on the WebSocket Server how many WebSockets clients can to connect?

[beyonlo]

Do you know if is possible to limit on the WebSocket Server how many WebSockets clients can to connect?

I did that just counting the ws objects appended on the dict, and after reach more than the max count, I just do ws.close() to end the WebSocket connection. I don't know if this approach is the best way to do that, so suggestions are welcome! :)

Thank you!

[miguelgrinberg]

Yes, that is a reasonable solution.

[miguelgrinberg]

@Carglglz Hey, I'm looking at your patch above and have a question. Why is it necessary to reset the SSLContext object with each request in your proposed implementation? This isn't in CPython's SSLContext, as far as I can see. Thanks!