plombery
plombery copied to clipboard
Duration timer never stops after 1st run
Tried a basic pipeline to check how return values are used in the GUI. The first run of the following simple code seems to generate unstable behaviour:
@task
def test():
return 10
@task
def test1():
return 20
@task
def test2():
return 30
register_pipeline(
id="test",
description="simple pipeline fails",
tasks=[test, test1, test2],
)
Here the output log:
INFO: Will watch for changes in these directories: ['/Users/luca/Documents/JobScraper']
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO: Started reloader process [33667] using StatReload
INFO: Started server process [33669]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: 127.0.0.1:60912 - "GET / HTTP/1.1" 304 Not Modified
INFO: 127.0.0.1:60912 - "GET /assets/index-86ef8bb9.js HTTP/1.1" 304 Not Modified
INFO: 127.0.0.1:60912 - "GET /assets/index-4fe84deb.css HTTP/1.1" 304 Not Modified
INFO: 127.0.0.1:60913 - "GET /assets/vendor-3fad8192.js HTTP/1.1" 304 Not Modified
INFO: 127.0.0.1:60913 - "GET /api/auth/whoami HTTP/1.1" 200 OK
INFO: 127.0.0.1:60913 - "GET /api/pipelines/ HTTP/1.1" 200 OK
INFO: ('127.0.0.1', 60914) - "WebSocket /api/ws/" [accepted]
INFO: connection open
INFO: 127.0.0.1:60913 - "GET /api/runs/?pipeline_id=&trigger_id= HTTP/1.1" 200 OK
INFO: 127.0.0.1:60912 - "GET /api/pipelines/ HTTP/1.1" 200 OK
INFO: 127.0.0.1:60913 - "GET /api/runs/?pipeline_id=&trigger_id= HTTP/1.1" 200 OK
INFO: 127.0.0.1:60916 - "GET /api/pipelines/ HTTP/1.1" 200 OK
INFO: 127.0.0.1:60915 - "GET /api/runs/?pipeline_id=&trigger_id= HTTP/1.1" 200 OK
INFO: 127.0.0.1:60915 - "GET /api/pipelines/test/input-schema HTTP/1.1" 200 OK
INFO: 127.0.0.1:60915 - "POST /api/runs/ HTTP/1.1" 200 OK
Executing pipeline `test` #1 via trigger `_manual`
Executing task test
Executing task test1
Executing task test2
INFO: 127.0.0.1:60915 - "GET /api/pipelines/test HTTP/1.1" 200 OK
INFO: ('127.0.0.1', 60917) - "WebSocket /api/ws/" [accepted]
Task exception was never retrieved
future: <Task finished name='Task-33' coro=<WebSocket.send_json() done, defined at /Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/starlette/websockets.py:168> exception=ConnectionClosedOK(Close(code=1005, reason=''), Close(code=1005, reason=''), True)>
Traceback (most recent call last):
File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/starlette/websockets.py", line 173, in send_json
await self.send({"type": "websocket.send", "text": text})
File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/starlette/websockets.py", line 85, in send
await self._send(message)
File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in sender
await send(message)
File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 320, in asgi_send
await self.send(data) # type: ignore[arg-type]
File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
await self.ensure_open()
File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/websockets/legacy/protocol.py", line 935, in ensure_open
raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: received 1005 (no status code [internal]); then sent 1005 (no status code [internal])
INFO: connection open
INFO: connection closed
INFO: 127.0.0.1:60916 - "GET /api/runs/1 HTTP/1.1" 200 OK
INFO: 127.0.0.1:60916 - "GET /api/runs/1/logs HTTP/1.1" 200 OK
INFO: 127.0.0.1:60918 - "GET /api/pipelines/test HTTP/1.1" 200 OK
INFO: 127.0.0.1:60920 - "GET /api/runs/1/logs HTTP/1.1" 200 OK
INFO: 127.0.0.1:60919 - "GET /api/runs/1 HTTP/1.1" 200 OK
UPDATE: it seems that adding a little delay into each task seems to fix problem, something like
time.sleep(0.2)
hi, thanks for reporting this, I'll check what's going on.
After some investigations on different OS and browsers I wasn't able to reproduce this issue, though it seems bound to the websocket and not to the pipeline executor, I'll try to handle those websocket connection issues!