How it works / Architecture page in documentation

[gherciu]

I assume a lot of people would love if this project had a (Architecture/ How It Works) page in the documentation. Because some of the architectural decisions may not be obvious.

Most of the items I'm interested to be documented are:

• Why are there 2 tables similar one named jobs and one archived, why archived is not just a flag like is_archived. Asking that because is kinda hard to tell which jobs failed because these go to archived and now we have to check 2 tables instead of one, I know I can increase the delay of a failed job to go to archived but Curious why it by default is structured like that.
• Another thing I would be interested in how it scales, and what are the benefits in using just simple async/await ower the child processes and threads and/or if is there plans to implement something like that.

[timgit]

In the docs I attempt to summarize the architecture, that being SKIP LOCKED combined with 1 or more job pollers, which favors a pull-based system instead of push. Also, the archive table is going away in v11, which should be released as soon as I can focus a bit more time on the schema migration. For scale, there are a lot of factors related to concurrency and total job volume, both of which I think have a much stronger impact over choosing how to run your pollers on a single server (processes, threading, etc).

[gherciu]

Thanks for your response, I would love to see what the next versions will bring Thanks for your time