zot
zot copied to clipboard
cleanup infra for background tasks
Is your feature request related to a problem? Please describe.
Here are the requirements for background tasks:
- Cannot aggressively hold storage locks since HTTP threads must be prioritized - interactive vs background jobs
- Single place to handle this.
- Make it easy for multiple extensions/features to use this.
- We expect embedded device use cases as well, so memory usage should be minimal
Describe the solution you'd like
Pls. look at: https://gobyexample.com/tickers https://github.com/go-co-op/gocron
Also, as an iterator over repos, the two options are:
- repos = get_all_repos(); foreach repo := repos { // do something }
The issue is that "repos" can become quite large and will this work on a raspberry pi, for example
- for repo = get_first_repo(); repo = get_next_repo(repo) { // do something }
A first/next iterator consumes more CPU potentially, but memory usage is small.
Also, always be clear about interactive and background tasks. Interactive tasks (HTTP threads) are highest priority and background tasks whenever possible must not degrade QoS for interactive tasks.