plombery icon indicating copy to clipboard operation
plombery copied to clipboard

Duration timer never stops after 1st run

Open lzzzam opened this issue 2 years ago • 3 comments

Tried a basic pipeline to check how return values are used in the GUI. The first run of the following simple code seems to generate unstable behaviour:

@task
def test():
    return 10

@task
def test1():
    return 20

@task
def test2():
    return 30

register_pipeline(
    id="test",
    description="simple pipeline fails",
    tasks=[test, test1, test2],
)

Here the output log:

INFO:     Will watch for changes in these directories: ['/Users/luca/Documents/JobScraper']
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [33667] using StatReload
INFO:     Started server process [33669]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     127.0.0.1:60912 - "GET / HTTP/1.1" 304 Not Modified
INFO:     127.0.0.1:60912 - "GET /assets/index-86ef8bb9.js HTTP/1.1" 304 Not Modified
INFO:     127.0.0.1:60912 - "GET /assets/index-4fe84deb.css HTTP/1.1" 304 Not Modified
INFO:     127.0.0.1:60913 - "GET /assets/vendor-3fad8192.js HTTP/1.1" 304 Not Modified
INFO:     127.0.0.1:60913 - "GET /api/auth/whoami HTTP/1.1" 200 OK
INFO:     127.0.0.1:60913 - "GET /api/pipelines/ HTTP/1.1" 200 OK
INFO:     ('127.0.0.1', 60914) - "WebSocket /api/ws/" [accepted]
INFO:     connection open
INFO:     127.0.0.1:60913 - "GET /api/runs/?pipeline_id=&trigger_id= HTTP/1.1" 200 OK
INFO:     127.0.0.1:60912 - "GET /api/pipelines/ HTTP/1.1" 200 OK
INFO:     127.0.0.1:60913 - "GET /api/runs/?pipeline_id=&trigger_id= HTTP/1.1" 200 OK
INFO:     127.0.0.1:60916 - "GET /api/pipelines/ HTTP/1.1" 200 OK
INFO:     127.0.0.1:60915 - "GET /api/runs/?pipeline_id=&trigger_id= HTTP/1.1" 200 OK
INFO:     127.0.0.1:60915 - "GET /api/pipelines/test/input-schema HTTP/1.1" 200 OK
INFO:     127.0.0.1:60915 - "POST /api/runs/ HTTP/1.1" 200 OK
Executing pipeline `test` #1 via trigger `_manual`
Executing task test
Executing task test1
Executing task test2
INFO:     127.0.0.1:60915 - "GET /api/pipelines/test HTTP/1.1" 200 OK
INFO:     ('127.0.0.1', 60917) - "WebSocket /api/ws/" [accepted]
Task exception was never retrieved
future: <Task finished name='Task-33' coro=<WebSocket.send_json() done, defined at /Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/starlette/websockets.py:168> exception=ConnectionClosedOK(Close(code=1005, reason=''), Close(code=1005, reason=''), True)>
Traceback (most recent call last):
  File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/starlette/websockets.py", line 173, in send_json
    await self.send({"type": "websocket.send", "text": text})
  File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/starlette/websockets.py", line 85, in send
    await self._send(message)
  File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in sender
    await send(message)
  File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 320, in asgi_send
    await self.send(data)  # type: ignore[arg-type]
  File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/websockets/legacy/protocol.py", line 935, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: received 1005 (no status code [internal]); then sent 1005 (no status code [internal])
INFO:     connection open
INFO:     connection closed
INFO:     127.0.0.1:60916 - "GET /api/runs/1 HTTP/1.1" 200 OK
INFO:     127.0.0.1:60916 - "GET /api/runs/1/logs HTTP/1.1" 200 OK
INFO:     127.0.0.1:60918 - "GET /api/pipelines/test HTTP/1.1" 200 OK
INFO:     127.0.0.1:60920 - "GET /api/runs/1/logs HTTP/1.1" 200 OK
INFO:     127.0.0.1:60919 - "GET /api/runs/1 HTTP/1.1" 200 OK

lzzzam avatar Oct 21 '23 10:10 lzzzam

UPDATE: it seems that adding a little delay into each task seems to fix problem, something like

time.sleep(0.2)

lzzzam avatar Oct 21 '23 10:10 lzzzam

hi, thanks for reporting this, I'll check what's going on.

lucafaggianelli avatar Oct 30 '23 17:10 lucafaggianelli

After some investigations on different OS and browsers I wasn't able to reproduce this issue, though it seems bound to the websocket and not to the pipeline executor, I'll try to handle those websocket connection issues!

lucafaggianelli avatar Oct 31 '23 10:10 lucafaggianelli