Add configuration to support free concurrent tasks and optimize git library update performance issues
In the process of frequently creating tasks in large batches, several problems were discovered:
- Concurrent tasks of the same template are not supported
- Taskpool is slow to deliver tasks every 5 seconds.
- The git library will need to be updated every time a task is executed, but in most cases this is not necessary.
This is controlled through the incremental scheme, and is switched by configuring GitCacheTime, TaskFreeConcurrencyMode, and TaskPoolTickerSec. When not configured, it remains the same as the previous code.
Consider that inventory is created through taskid when using git. It is actually created every time. Currently, GitCacheTime does not control this.
Hi @gaoyue1989
Did you test the PR in real tasks?
Caching doesn't resolve the conflict issue and introduces unexpected behavior.
I think we need to check for new commits (without pulling) on each task run.
In free concurrency mode, new tasks fail if a new commit appears. Or ignore new commits if active tasks exist. We can add numeric option for this: max_skip_attempts_for_new_commits (name must be changed :D).