Support pyodide

[chrysn]

Most of aiocoap can run quite fine in pyodide (Python running in the browser by compilation to WASM through emscripten). They provide asyncio support through their webloop module, so things should be quite fine overall.

Trouble is of course, a browser can't connect, bind, accept or getaddrinfo -- but that's what we have RFC8323 for.

The websockets module currently providing coap+ws connectivity does not work in pyodide because it's not pure, but wouldn't help anyway because what it does is implement WS based on sockets. But pyodide exposes websockets (may need a shim layer, though).

Rough steps:

• Determine if this would be useful. (Might be for quick demos, but might also need better proxy support to allow setting things up easily with a forward proxy going from ws to the rest of the world).
• Ensure aiocoap can be loaded into pyodide (by uploading bdist_wheel --universal)
• Add a transport for pyodide websockets (possibly using a drop-in-replacement shim for the real websockets module that can do just the minimum needed)
• Find a way to test this
• Integrate it into examples

[chrysn]

Current as-far-as-I-get code:

import micropip
await micropip.install('aiocoap[oscore]')
import aiocoap
ctx = await aiocoap.Context.create_client_context()
req = ctx.request(aiocoap.Message(code=aiocoap.GET, uri="coap+ws://demo.coap.amsuess.com/.well-known/core"))
res = await req.response

which fails with "no transport can route" (obviously, because I didn't install websockets as that's failing) or NotImplemented when picking one of the other transports. (By the way, kudos to pyodide implementers for not going the emscripten way here and opening a websocket to RPC POSIX sockets here -- NotImplemented is the right choice here IMO).

[chrysn]

On how this could be used:

• With probably just one change (set the forward proxy), all the guided tour and examples could be run without local installation.
• With RD proxy registration, this might be convenient for introductory demo servers, especially in classroom environments where network access is limited. (Students can have a tabbed code-and-log view, change the code, restart the server, and run aiocoap-client against other students' servers).
• Just giving an online aiocoap-client (especially if it becomes better in #235)

Probably not for:

• copper-style browser experience. (Possible technically, but I interfacing a HTML GUI and Python is probably awkward for all involved).

[chrysn]

Since the latest release, wheels are uploaded, so the install is just await micropip.install('aiocoap') now.

[chrysn]

A blocker to any serious use is https://github.com/pyodide/pyodide/issues/761, as without cryptography there is no OSCORE. (Sure there can be wss security using the browser trust model, but I don't want to build a proxy that unwraps OSCORE for the client because the client can't do it, it'd undermine the E2E promise).

[chrysn]

Things are a bit better now:

• The cryptography module is now available; only cbor2 is still missing when installing with [oscore]
• Released versions of aiocoap now ship suitable wheels

Main issue is still the lack of a backend based on browser websockets.

[chrysn]

A new setback is the upcoming cbor-diag dependency of #302.

[chrysn]

Seems like websockets are straightforward enough -- and starboard might be a good demo platform, given it just works with asyncio.

Minimal scaffolding:

• go to https://starboard.gg/
• At the top, insert an HTML snip and run it:

<div id="clickme">Initial Text</div>

• Then, add a Python snip and run it:

from js import document as dom
clickme = dom.getElementById('clickme')

import micropip
await micropip.install('aiocoap')
import aiocoap
# doing full WS is too much, but let's do the minimal mixing of JS and asyncio
import asyncio
queue = asyncio.Queue()
def clicked(evt):
    print("event", evt)
    queue.put_nowait("clicked")
clickme.addEventListener('click', clicked)
print(aiocoap.Message, clickme)
await queue.get()

[chrysn]

more websock-ish, inspired by https://starboard.gg/ACBIAS/Demo-Websockets-Pyodide-nbamPAL:

import micropip
await micropip.install('aiocoap')
import aiocoap
import js
GetWebSocket = js.eval('(function() {return new WebSocket(...arguments);})')
import asyncio
queue = asyncio.Queue()
socket = GetWebSocket("wss://proxy.coap.amsuess.com/.well-known/coap")
socket.addEventListener("open", lambda e: queue.put_nowait("open"))
socket.addEventListener("close", lambda e: queue.put_nowait("close"))
socket.addEventListener("message", lambda e: queue.put_nowait(e))
async def run(queue, socket):
  print(await queue.get())
  socket.send(b"\x00")
  while True:
    event = await queue.get()
    if isinstance(event, str):
      print(event)
    else:
      print("data", repr(event.data))
    if event == "close":
      return
await run(queue, socket)

Note that this only works on starboard which still uses pyodide 0.18.1; on an actual pyodide console that runs 0.22.1 at the moment, we'll need more explicit proxying.

[chrysn]

For pyodide 0.22.1 (eg. their online console), you need to use add_event_listeners from pyodide's FFI wrappers (as I understand, the old style leaked memory, and after the leak was fixed there was a need for event handling that's a bit more aware of what is happening). There, run:

import micropip
await micropip.install('aiocoap')
import aiocoap
import js
GetWebSocket = js.eval('(function() {return new WebSocket(...arguments);})')
import asyncio
queue = asyncio.Queue()
socket = GetWebSocket("wss://proxy.coap.amsuess.com/.well-known/coap")
from pyodide.ffi.wrappers import add_event_listener
add_event_listener(socket, "open", lambda e: queue.put_nowait("open"))
add_event_listener(socket, "close", lambda e: queue.put_nowait("close"))
add_event_listener(socket, "message", lambda e: queue.put_nowait(e))
async def run(queue, socket):
  print(await queue.get())
  socket.send(js.Blob.new([js.Uint8Array.new(b"\x00\xff")]))
  while True:
    event = await queue.get()
    if isinstance(event, str):
      print(event)
    else:
      buffer = (await event.data.arrayBuffer()).to_py()
      print("data", bytes(buffer))
    if event == "close":
      return

and run await run(queue, socket) in an extra line (because pyodide's shell does top-level awaits different than starboard does).

[chrysn]

As far as more notebook-style interpreters go, https://jupyter.org/try-jupyter/lab/ is an option