proletarian
proletarian copied to clipboard
msolli
A durable job queuing and worker system for Clojure backed by PostgreSQL.
I noticed that when an assert statement fails in one of my jobs the whole proletarian Queue Worker shuts down. A failing assertion generates an `AssertionError` which is not a...
Just noticed this warning: ``` Reflection warning, proletarian/worker.clj:235:16 - call to static method sleep on java.lang.Thread can't be resolved (argument types: java.lang.Number). ``` The reason is probably because `Thread` in...
Hello @msolli, thanks for your work on this library. Did you think about adding monitoring/instrumentation to the jobs? Although I understand it might be possible to wrap the handle-job! multimethod...
Running workers on machines in different timezones confuses proletarian because it stores `process_at` in the db as a local timestamp w/o timezone. Perhaps it would be better if the timestamps...
Hello! is there any reason of not passing job-id besides job-type to the worker handler-fn? I think it could be useful to identify the job execution context with this uuid...
We need to write the `proletarian.job` table outside of Clojure, therefore I switched the serializer from Transit to JSON (Jsonista). Everything worked to the point where I switched the `payload`...
Is there some way of getting a notification when a job is done? For instance: - start a long running job, like a couple of minutes - when the job...
Error `FATAL: terminating connection due to idle-in-transaction timeout\n\tat org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2676)\n\tat org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2366)\n\tat org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:356)\n\tat org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:496)\n\tat org.postgresql.jdbc.PgStatement.execute(PgStatement.java:413)\n\tat org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:190)\n\tat org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:152)\n\tat com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)\n\tat com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)\n\tat proletarian.db$archive_job_BANG_.invokeStatic(db.clj:87)\n\tat proletarian.worker$process_next_job_BANG_$fn__48219$fn__48221.invoke(worker.clj:57)\n\tat proletarian.worker$process_next_job_BANG_$fn__48219.invoke(worker.clj:53)\n\tat proletarian.db$with_tx$fn__39340.invoke(db.clj:139)\n\tat proletarian.db$with_connection.invokeStatic(db.clj:129)\n\tat proletarian.db$with_tx.invokeStatic(db.clj:131)\n\tat proletarian.worker$process_next_job_BANG_.invokeStatic(worker.clj:12)\n\tat proletarian.worker$process_next_jobs_BANG_.invokeStatic(worker.clj:81)\n\tat proletarian.worker$create_queue_worker$reify__48237$work_BANG___48243.invoke(worker.clj:208)\n\tat clojure.core$partial$fn__5908.invoke(core.clj:2640)\n\tat clojure.lang.AFn.run(AFn.java:22)\n\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)\n\tat java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)\n\tat...
Hey! `process-next-job!` seems to [run in a transaction](https://github.com/msolli/proletarian/blob/main/src/proletarian/worker.clj#L53), if `failed-job-fn` also throws an exception (which runs in that transaction), the database changes to clean up the job will be rolled...
I have a handful of queues running and every few days one of them will throw this exception: ``` #error { :cause "ERROR: invalid input syntax for type timestamp: \"2025-04-29...
