zot icon indicating copy to clipboard operation
zot copied to clipboard

cleanup infra for background tasks

Open rchincha opened this issue 2 years ago • 2 comments

Is your feature request related to a problem? Please describe.

Here are the requirements for background tasks:

  1. Cannot aggressively hold storage locks since HTTP threads must be prioritized - interactive vs background jobs
  2. Single place to handle this.
  3. Make it easy for multiple extensions/features to use this.
  4. We expect embedded device use cases as well, so memory usage should be minimal

Describe the solution you'd like

Pls. look at: https://gobyexample.com/tickers https://github.com/go-co-op/gocron

rchincha avatar Jun 02 '22 16:06 rchincha

Also, as an iterator over repos, the two options are:

  1. repos = get_all_repos(); foreach repo := repos { // do something }

The issue is that "repos" can become quite large and will this work on a raspberry pi, for example

  1. for repo = get_first_repo(); repo = get_next_repo(repo) { // do something }

A first/next iterator consumes more CPU potentially, but memory usage is small.

rchincha avatar Jul 20 '22 20:07 rchincha

Also, always be clear about interactive and background tasks. Interactive tasks (HTTP threads) are highest priority and background tasks whenever possible must not degrade QoS for interactive tasks.

rchincha avatar Jul 20 '22 20:07 rchincha