Ranked statistics periodic caching

[stveit]

Fixes #1504

Adds a system for periodic caching of ranked statistics. Specific reports that should be periodically cached can be set in config file.

Hardcoded how often caches are run for hour / daily / week / month reports. Necessary to make this dynamic so user can specify how often to run these? Any good ways to have dynamic schedule for cron job?

How long should cache timeout be? how often should hour / daily / etc. cron job be run? Cache timeout should at least be a bit longer than time between cron jobs. say 15 min time between hourly reports, if timeout is 15 min then there might be some moments with an empty cache if graphite is randomly a bit slower and a cron job finishes more than 15 minutes after the previous one finished.

How should this be reflected in front end? Currently theres a true/false box that when toggled will attempt to read from cache. If cache is empty or the box is toggled off, then it gets the data from graphite directly and updates the cache.

[github-actions[bot]]

Test results

     12 files       12 suites   12m 44s :stopwatch: 3 216 tests 3 120 :heavy_check_mark:   96 :zzz: 0 :x: 9 123 runs  8 835 :heavy_check_mark: 288 :zzz: 0 :x:

Results for commit 54c28e68.

:recycle: This comment has been updated with latest results.

[sonarqubecloud[bot]]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[stveit]

Force push to fix the commit log that went out of control. There have also been made some changes to the UI to better reflect if cache is used or not, including the time it took to get the requested data (looks a bit debuggy the way its shown atm)

[sonarqubecloud[bot]]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
1 Code Smell

No Coverage information
0.0% Duplication

[codecov[bot]]

Codecov Report

Merging #2398 (54c28e6) into master (6f1f637) will increase coverage by 0.05%. The diff coverage is 72.81%.

@@            Coverage Diff             @@
##           master    #2398      +/-   ##
==========================================
+ Coverage   53.54%   53.60%   +0.05%     
==========================================
  Files         556      557       +1     
  Lines       40432    40521      +89     
==========================================
+ Hits        21651    21722      +71     
- Misses      18781    18799      +18     

Impacted Files	Coverage Δ
python/nav/django/settings.py	97.67% <ø> (ø)
python/nav/web/sortedstats/statmodules.py	41.53% <9.09%> (-0.65%)	:arrow_down:
python/nav/web/sortedstats/views.py	64.17% <73.46%> (+27.81%)	:arrow_up:
python/nav/web/sortedstats/forms.py	74.07% <75.00%> (-0.93%)	:arrow_down:
python/nav/web/sortedstats/config.py	89.18% <89.18%> (ø)
python/nav/web/sortedstats/__init__.py	100.00% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[lunkwill42]

Note to self: This may need an alternative approach, since the problem is about requests timing out

[sonarqubecloud[bot]]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication