Closes #400

Note that django-q2 won't even install on python 3.7.

Dependencies

There's a new one, so remember to install django-q2 in the venv.

Database

Depending on django_q2 adds new database tables, so migrations must be run:

python manage.py migrate

Redis

It currently uses redis, the same redis that is configured for websockets, so set the environment-variable ARGUS_REDIS_SERVER. This is used to make the redis-settings in src/argus/site/settings/base.py, setting Q_CLUSTER. Override in local settings if necessary.

Running django-q2

Start the brokers with

python manage.py qcluster

It does not run in the background and can be turned off with Ctrl-C. Note: if any changes are made to the code, qcluster must be restarted, it does not automatically discover changes.

Check if a task worked or failed in Django Admin, the "Django Q2 Queue"-section.

Test sending notifications

Test queuing notifications by

Disabling profiles sending to non-email destinations
Ensuring that EMAIL_BACKEND is 'django.core.mail.backends.console.EmailBackend', this pumps the contents of the email to the console/sys.stdout where qcluster is running.
python manage.py create_fake_incident

In the terminal where you do step 3, the last log-line will contain something like: "Notification: will be sending notification for "'Incident start':..".

In the terminal where qcluster is running, you'll get a log line starting with "Notification: sending event "'Incident start':..". The header and body will be dumped as plaintext after this line. After the email there will be a log line starting with "Notification: sent event "'Incident start':..".

Docker

The postgres image has been updated to version 13, this means the volume should be deleted and remade. Clean everything away with:

docker compose down --volumes

Apr 25 '23 07:04 hmpf

Test results

      7 files   574 suites 21m 39s :stopwatch:   462 tests   461 :heavy_check_mark: 1 :zzz: 0 :x: 3 234 runs 3 227 :heavy_check_mark: 7 :zzz: 0 :x:

Results for commit 4fe29601.

:recycle: This comment has been updated with latest results.

Apr 25 '23 07:04 github-actions[bot]

Missing:

docs (how to)
work with bulk-fix
update docker this-and-that

Apr 25 '23 07:04 hmpf

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
3 Code Smells

No Coverage information
0.0% Duplication

Apr 27 '23 08:04 sonarqubecloud[bot]

Codecov Report

Attention: Patch coverage is 73.07692% with 7 lines in your changes are missing coverage. Please review.

Project coverage is 84.70%. Comparing base (6488afa) to head (4fe2960).

Files	Patch %	Lines
src/argus/incident/signals.py	56.25%	7 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #630      +/-   ##
==========================================
+ Coverage   84.61%   84.70%   +0.09%     
==========================================
  Files          75       75              
  Lines        3750     3754       +4     
==========================================
+ Hits         3173     3180       +7     
+ Misses        577      574       -3

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Sep 20 '23 06:09 codecov-commenter

I added an extra qcluster service to docker-compose.yml, which basically is a copy of api, but which executes the manage.py qcluster command rather than daphne, and can get it to dispatch notifications according to my profile when I inject fake ones.

i.e. it's usable, but I don't like the redundant environment definitions in docker-compose.yml - I'm sure there some way we can template those out, though.

Sep 20 '23 13:09 lunkwill42

I've looked at bulk-changes (Incident.objects.create_events()).

Should we make one notification per change (easy, by putting the new events on the queue one at a time) or should we make a single notification for a "bulk-change"-event? The problem with the latter is matching it with filters: a filter might hit one of the events in the bulk, but not all of them.

Sep 21 '23 06:09 hmpf

I've looked at bulk-changes (Incident.objects.create_events()).

Should we make one notification per change (easy, by putting the new events on the queue one at a time) or should we make a single notification for a "bulk-change"-event? The problem with the latter is matching it with filters: a filter might hit one of the events in the bulk, but not all of them.

I like the easy and less complicated route of one queued job per notification. It shouldn't become a bottleneck, but if it does, we can work it out later.

Sep 21 '23 09:09 lunkwill42

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
3 Code Smells