procrastinate icon indicating copy to clipboard operation
procrastinate copied to clipboard

Help with periodic tasks

Open alucab1 opened this issue 4 years ago • 7 comments

Hi, I'm trying to write code that schedules a task to be executed every few seconds. However, in my console, I can see all the tasks being defered, but they are never executed.

import procrastinate
#from procrastinate import AiopgConnector, App
import time
import sys
import asyncio
from procrastinate import AiopgConnector, App

app = App(
    connector=AiopgConnector(host="localhost", user="postgres", password="password")
)
app.open()

# Define tasks using coroutine functions
@app.periodic(cron="*/30 * * * * *")
@app.task(
    name="print analysis",
    queue="power",
    lock="power"
)
def power(timestamp: int):
    print("4")
    return "4"


if __name__ == "__main__":
    print("D")
    app.run_worker(wait=False)`

I tried adding a second worker, but then that introduced a bunch of errors.

C:\Users\cabrea2\AppData\Roaming\Python\Python39\site-packages\aiopg\pool.py:392: ResourceWarning: Invalid transaction status on released connection: 1
  warnings.warn(
Traceback (most recent call last):
  File "C:\Users\cabrea2\git\gitini\scheduler.py", line 29, in <module>
    app.run_worker(wait=False, name="executer")
  File "C:\Users\cabrea2\AppData\Roaming\Python\Python39\site-packages\procrastinate\utils.py", line 118, in wrapper
    return sync_await(awaitable=awaitable)
  File "C:\Users\cabrea2\AppData\Roaming\Python\Python39\site-packages\procrastinate\utils.py", line 168, in sync_await
    return loop.run_until_complete(awaitable)
  File "C:\Python39\lib\asyncio\base_events.py", line 642, in run_until_complete
    return future.result()
  File "C:\Users\cabrea2\AppData\Roaming\Python\Python39\site-packages\procrastinate\app.py", line 320, in run_worker_async
    await worker.run()
  File "C:\Users\cabrea2\AppData\Roaming\Python\Python39\site-packages\procrastinate\worker.py", line 124, in run
    await asyncio.gather(
  File "C:\Users\cabrea2\AppData\Roaming\Python\Python39\site-packages\procrastinate\worker.py", line 140, in single_worker
    job = await self.job_manager.fetch_job(self.queues)
  File "C:\Users\cabrea2\AppData\Roaming\Python\Python39\site-packages\procrastinate\manager.py", line 117, in fetch_job
    row = await self.connector.execute_query_one_async(
  File "C:\Users\cabrea2\AppData\Roaming\Python\Python39\site-packages\procrastinate\aiopg_connector.py", line 31, in wrapped
    return await coro(*args, **kwargs)
  File "C:\Users\cabrea2\AppData\Roaming\Python\Python39\site-packages\procrastinate\aiopg_connector.py", line 67, in wrapped
    return await coro(*args, **kwargs)
  File "C:\Users\cabrea2\AppData\Roaming\Python\Python39\site-packages\procrastinate\aiopg_connector.py", line 269, in execute_query_one_async
    await cursor.execute(query, self._wrap_json(arguments))
  File "C:\Users\cabrea2\AppData\Roaming\Python\Python39\site-packages\aiopg\connection.py", line 432, in execute
    await self._conn._poll(waiter, timeout)
  File "C:\Users\cabrea2\AppData\Roaming\Python\Python39\site-packages\aiopg\connection.py", line 887, in _poll
    await asyncio.wait_for(self._waiter, timeout)
  File "C:\Python39\lib\asyncio\tasks.py", line 462, in wait_for
    fut = ensure_future(fut, loop=loop)
  File "C:\Python39\lib\asyncio\tasks.py", line 674, in ensure_future
    raise ValueError('The future belongs to a different loop than '
ValueError: The future belongs to a different loop than the one specified as the loop argument

Any help is appreciated, I need to finish this code for my internship.

alucab1 avatar Sep 21 '21 22:09 alucab1

@alucab1 can you please use triple backticks around your code blocks for a proper formatting of your code? Example:

@app.periodic(cron="*/30 * * * * *")
@app.task(
    name="print analysis",
    queue="power",
    lock="power"
)
def power(timestamp: int):
    print("4")
    return "4"

This would make your code much more readable. The ideal is that we just need to copy your run, paste into a file, and run that file to reproduce your problem.

elemoine avatar Sep 22 '21 05:09 elemoine

Alright, sorry about that. I edited the post.

alucab1 avatar Sep 22 '21 13:09 alucab1

Also, for the postgreSQL datatbase I just created a temporary one in docker with the command provided in the Quickstart manual: docker run --name pg-procrastinate --detach --rm -p 5432:5432 -e POSTGRES_PASSWORD=password postgres

alucab1 avatar Sep 22 '21 14:09 alucab1

I'm not sure we support Windows :/ I know we have seen problems before with windows hosts. There's no one on the team with either windows knowledge or a windows computer to run the tests. Last time it was discussed was in #286 and #444

ewjoachim avatar Sep 22 '21 15:09 ewjoachim

@alucab1 if you're interested in contributing a fix to use (and run tests under) windows, I think we'd merge it :)

ewjoachim avatar Sep 22 '21 15:09 ewjoachim

Hm, just a hunch... Why do you have wait=False in run_worker ? Have you tried removing this and see if it works ?

ewjoachim avatar Sep 27 '21 08:09 ewjoachim

@alucab1 Hey :) Is this still an issue ?

ewjoachim avatar Dec 23 '21 13:12 ewjoachim

Closed for lack of response. Feel free to reopen if relevant.

ewjoachim avatar Feb 23 '23 09:02 ewjoachim