Celery monitors

[york-stsci]

This will alter the existing monitors (as little as possible) by replacing the calls to the pipeline_tools calibration function with a blocking call to the celery calibration task. As such, code flow will work exactly as it would before, except that (of course) instead of waiting for the pipeline to run, the code will instead wait for the pipeline task to move up the queue, and then run.

Closes #905 Closes #907

[pep8speaks]

Hello @york-stsci, Thank you for updating !

• In the file jwql/instrument_monitors/common_monitors/bad_pixel_monitor.py:

Line 805:1: W293 blank line contains whitespace Line 855:85: W291 trailing whitespace Line 856:48: W291 trailing whitespace

• In the file jwql/instrument_monitors/common_monitors/bias_monitor.py:

Line 372:1: W293 blank line contains whitespace Line 379:1: W293 blank line contains whitespace

• In the file jwql/instrument_monitors/common_monitors/dark_monitor.py:

Line 89:1: E302 expected 2 blank lines, found 1 Line 494:1: W293 blank line contains whitespace Line 515:1: W293 blank line contains whitespace Line 594:1: W293 blank line contains whitespace Line 800:39: E127 continuation line over-indented for visual indent

• In the file jwql/instrument_monitors/common_monitors/readnoise_monitor.py:

Line 358:61: E226 missing whitespace around arithmetic operator Line 358:110: E226 missing whitespace around arithmetic operator Line 361:61: E226 missing whitespace around arithmetic operator Line 361:110: E226 missing whitespace around arithmetic operator Line 373:33: E226 missing whitespace around arithmetic operator Line 413:1: W293 blank line contains whitespace Line 435:13: E265 block comment should start with '# ' Line 436:13: E265 block comment should start with '# ' Line 605:67: E226 missing whitespace around arithmetic operator Line 605:70: E226 missing whitespace around arithmetic operator Line 607:66: E226 missing whitespace around arithmetic operator

• In the file jwql/shared_tasks/shared_tasks.py:

Line 41:1: W293 blank line contains whitespace Line 56:1: W293 blank line contains whitespace Line 63:1: W293 blank line contains whitespace Line 121:89: W291 trailing whitespace Line 127:88: W291 trailing whitespace Line 129:87: W291 trailing whitespace Line 136:85: W291 trailing whitespace Line 138:88: W291 trailing whitespace Line 178:1: E302 expected 2 blank lines, found 1 Line 193:1: E302 expected 2 blank lines, found 1 Line 220:1: W293 blank line contains whitespace Line 225:89: W291 trailing whitespace Line 226:87: W291 trailing whitespace Line 227:83: W291 trailing whitespace Line 239:1: W293 blank line contains whitespace Line 244:1: W293 blank line contains whitespace Line 249:1: W293 blank line contains whitespace Line 255:1: W293 blank line contains whitespace Line 357:1: W293 blank line contains whitespace Line 364:1: W293 blank line contains whitespace Line 405:1: W293 blank line contains whitespace Line 442:1: W293 blank line contains whitespace Line 453:1: W293 blank line contains whitespace Line 454:86: W291 trailing whitespace Line 458:1: W293 blank line contains whitespace Line 460:1: W293 blank line contains whitespace Line 465:1: W293 blank line contains whitespace Line 468:1: W293 blank line contains whitespace Line 473:1: W293 blank line contains whitespace Line 476:1: W293 blank line contains whitespace Line 485:1: W293 blank line contains whitespace Line 496:1: W293 blank line contains whitespace Line 499:1: W293 blank line contains whitespace Line 501:40: E226 missing whitespace around arithmetic operator Line 517:1: W293 blank line contains whitespace Line 519:1: W293 blank line contains whitespace Line 521:1: W293 blank line contains whitespace Line 523:1: W293 blank line contains whitespace Line 528:1: W293 blank line contains whitespace Line 529:88: W291 trailing whitespace Line 530:89: W291 trailing whitespace Line 532:1: W293 blank line contains whitespace Line 534:1: W293 blank line contains whitespace Line 545:1: W293 blank line contains whitespace Line 548:1: W293 blank line contains whitespace Line 576:1: W293 blank line contains whitespace Line 580:1: W293 blank line contains whitespace Line 588:1: W293 blank line contains whitespace Line 591:1: W293 blank line contains whitespace Line 595:55: W291 trailing whitespace Line 604:1: W293 blank line contains whitespace Line 613:55: E226 missing whitespace around arithmetic operator Line 613:106: E226 missing whitespace around arithmetic operator Line 622:87: W291 trailing whitespace Line 624:1: W293 blank line contains whitespace Line 629:87: W291 trailing whitespace Line 633:1: W293 blank line contains whitespace Line 634:83: W291 trailing whitespace Line 644:1: W293 blank line contains whitespace Line 647:1: W293 blank line contains whitespace Line 672:1: W293 blank line contains whitespace Line 682:1: W293 blank line contains whitespace Line 687:87: W291 trailing whitespace Line 691:1: W293 blank line contains whitespace Line 692:83: W291 trailing whitespace Line 699:1: W293 blank line contains whitespace Line 705:1: W293 blank line contains whitespace Line 708:86: W291 trailing whitespace Line 711:1: W293 blank line contains whitespace Line 714:1: W293 blank line contains whitespace Line 726:1: W293 blank line contains whitespace Line 764:1: W293 blank line contains whitespace

• In the file jwql/utils/logging_functions.py:

Line 243:99: E261 at least two spaces before inline comment

• In the file jwql/utils/utils.py:

Line 527:86: W504 line break after binary operator Line 528:79: W504 line break after binary operator Line 534:80: W504 line break after binary operator Line 535:79: W504 line break after binary operator

• In the file jwql/website/apps/jwql/data_containers.py:

Line 52:34: E126 continuation line over-indented for hanging indent Line 74:1: E402 module level import not at top of file Line 75:1: E402 module level import not at top of file Line 214:64: E226 missing whitespace around arithmetic operator Line 974:17: W503 line break before binary operator Line 975:17: W503 line break before binary operator Line 976:17: W503 line break before binary operator Line 977:17: W503 line break before binary operator Line 1327:9: E722 do not use bare 'except'

• In the file jwql/website/apps/jwql/views.py:

Line 305:9: E303 too many blank lines (2) Line 322:13: E265 block comment should start with '# '

Comment last updated at 2022-09-09 23:48:47 UTC

[york-stsci]

I have verified that the bad pixel monitor will submit calibration requests, take the outputs, and produce its final products. I'll add comments like this for each of the monitors that call the pipeline, although with less detail now that I've established the template.

[york-stsci]

Bias monitor runs to completion without crashes.

[york-stsci]

Dark and Readnoise monitors run through at least a single mode and save to database without problems. Did not have time to run through the entire dataset.

[mfixstsci]

Hello @york-stsci, Thank you for updating !

* In the file [`jwql/instrument_monitors/common_monitors/bad_pixel_monitor.py`](https://github.com/spacetelescope/jwql/blob/763cd1c5cbb56d57e30822dd5b15c56c712eda00/jwql/instrument_monitors/common_monitors/bad_pixel_monitor.py):

Line 805:1: W293 blank line contains whitespace Line 855:85: W291 trailing whitespace Line 856:48: W291 trailing whitespace

* In the file [`jwql/instrument_monitors/common_monitors/bias_monitor.py`](https://github.com/spacetelescope/jwql/blob/763cd1c5cbb56d57e30822dd5b15c56c712eda00/jwql/instrument_monitors/common_monitors/bias_monitor.py):

Line 372:1: W293 blank line contains whitespace Line 379:1: W293 blank line contains whitespace

* In the file [`jwql/instrument_monitors/common_monitors/dark_monitor.py`](https://github.com/spacetelescope/jwql/blob/763cd1c5cbb56d57e30822dd5b15c56c712eda00/jwql/instrument_monitors/common_monitors/dark_monitor.py):

Line 89:1: E302 expected 2 blank lines, found 1 Line 494:1: W293 blank line contains whitespace Line 515:1: W293 blank line contains whitespace Line 594:1: W293 blank line contains whitespace Line 800:39: E127 continuation line over-indented for visual indent

* In the file [`jwql/instrument_monitors/common_monitors/readnoise_monitor.py`](https://github.com/spacetelescope/jwql/blob/763cd1c5cbb56d57e30822dd5b15c56c712eda00/jwql/instrument_monitors/common_monitors/readnoise_monitor.py):

Line 358:61: E226 missing whitespace around arithmetic operator Line 358:110: E226 missing whitespace around arithmetic operator Line 361:61: E226 missing whitespace around arithmetic operator Line 361:110: E226 missing whitespace around arithmetic operator Line 373:33: E226 missing whitespace around arithmetic operator Line 413:1: W293 blank line contains whitespace Line 435:13: E265 block comment should start with '# ' Line 436:13: E265 block comment should start with '# ' Line 605:67: E226 missing whitespace around arithmetic operator Line 605:70: E226 missing whitespace around arithmetic operator Line 607:66: E226 missing whitespace around arithmetic operator

* In the file [`jwql/shared_tasks/shared_tasks.py`](https://github.com/spacetelescope/jwql/blob/763cd1c5cbb56d57e30822dd5b15c56c712eda00/jwql/shared_tasks/shared_tasks.py):

Line 41:1: W293 blank line contains whitespace Line 56:1: W293 blank line contains whitespace Line 63:1: W293 blank line contains whitespace Line 121:89: W291 trailing whitespace Line 127:88: W291 trailing whitespace Line 129:87: W291 trailing whitespace Line 136:85: W291 trailing whitespace Line 138:88: W291 trailing whitespace Line 178:1: E302 expected 2 blank lines, found 1 Line 193:1: E302 expected 2 blank lines, found 1 Line 220:1: W293 blank line contains whitespace Line 225:89: W291 trailing whitespace Line 226:87: W291 trailing whitespace Line 227:83: W291 trailing whitespace Line 239:1: W293 blank line contains whitespace Line 244:1: W293 blank line contains whitespace Line 249:1: W293 blank line contains whitespace Line 255:1: W293 blank line contains whitespace Line 357:1: W293 blank line contains whitespace Line 364:1: W293 blank line contains whitespace Line 405:1: W293 blank line contains whitespace Line 442:1: W293 blank line contains whitespace Line 453:1: W293 blank line contains whitespace Line 454:86: W291 trailing whitespace Line 458:1: W293 blank line contains whitespace Line 460:1: W293 blank line contains whitespace Line 465:1: W293 blank line contains whitespace Line 468:1: W293 blank line contains whitespace Line 473:1: W293 blank line contains whitespace Line 476:1: W293 blank line contains whitespace Line 485:1: W293 blank line contains whitespace Line 496:1: W293 blank line contains whitespace Line 499:1: W293 blank line contains whitespace Line 501:40: E226 missing whitespace around arithmetic operator Line 517:1: W293 blank line contains whitespace Line 519:1: W293 blank line contains whitespace Line 521:1: W293 blank line contains whitespace Line 523:1: W293 blank line contains whitespace Line 528:1: W293 blank line contains whitespace Line 529:88: W291 trailing whitespace Line 530:89: W291 trailing whitespace Line 532:1: W293 blank line contains whitespace Line 534:1: W293 blank line contains whitespace Line 545:1: W293 blank line contains whitespace Line 548:1: W293 blank line contains whitespace Line 576:1: W293 blank line contains whitespace Line 580:1: W293 blank line contains whitespace Line 588:1: W293 blank line contains whitespace Line 591:1: W293 blank line contains whitespace Line 595:55: W291 trailing whitespace Line 604:1: W293 blank line contains whitespace Line 613:55: E226 missing whitespace around arithmetic operator Line 613:106: E226 missing whitespace around arithmetic operator Line 622:87: W291 trailing whitespace Line 624:1: W293 blank line contains whitespace Line 629:87: W291 trailing whitespace Line 633:1: W293 blank line contains whitespace Line 634:83: W291 trailing whitespace Line 644:1: W293 blank line contains whitespace Line 647:1: W293 blank line contains whitespace Line 672:1: W293 blank line contains whitespace Line 682:1: W293 blank line contains whitespace Line 687:87: W291 trailing whitespace Line 691:1: W293 blank line contains whitespace Line 692:83: W291 trailing whitespace Line 699:1: W293 blank line contains whitespace Line 705:1: W293 blank line contains whitespace Line 708:86: W291 trailing whitespace Line 711:1: W293 blank line contains whitespace Line 714:1: W293 blank line contains whitespace Line 726:1: W293 blank line contains whitespace Line 764:1: W293 blank line contains whitespace

* In the file [`jwql/utils/logging_functions.py`](https://github.com/spacetelescope/jwql/blob/763cd1c5cbb56d57e30822dd5b15c56c712eda00/jwql/utils/logging_functions.py):

Line 243:99: E261 at least two spaces before inline comment

* In the file [`jwql/utils/utils.py`](https://github.com/spacetelescope/jwql/blob/763cd1c5cbb56d57e30822dd5b15c56c712eda00/jwql/utils/utils.py):

Line 527:86: W504 line break after binary operator Line 528:79: W504 line break after binary operator Line 534:80: W504 line break after binary operator Line 535:79: W504 line break after binary operator

* In the file [`jwql/website/apps/jwql/data_containers.py`](https://github.com/spacetelescope/jwql/blob/763cd1c5cbb56d57e30822dd5b15c56c712eda00/jwql/website/apps/jwql/data_containers.py):

Line 52:34: E126 continuation line over-indented for hanging indent Line 74:1: E402 module level import not at top of file Line 75:1: E402 module level import not at top of file Line 214:64: E226 missing whitespace around arithmetic operator Line 974:17: W503 line break before binary operator Line 975:17: W503 line break before binary operator Line 976:17: W503 line break before binary operator Line 977:17: W503 line break before binary operator Line 1327:9: E722 do not use bare 'except'

* In the file [`jwql/website/apps/jwql/views.py`](https://github.com/spacetelescope/jwql/blob/763cd1c5cbb56d57e30822dd5b15c56c712eda00/jwql/website/apps/jwql/views.py):

Line 305:9: E303 too many blank lines (2) Line 322:13: E265 block comment should start with '# '

Comment last updated at 2022-09-09 23:48:47 UTC

@york-stsci thank you for addressing all of our comments. This looks great. One last thing before we can merge, we need to tackle the pep8 fixes. In the past we have had some issues with @pep8speaks updating when corrections have been made. Let me know if that's the case for you as well.