maestrowf
maestrowf copied to clipboard
LLNL
Feature/crkrenn/summary status
@FrankD412 @jsemler
This is a very simple extension to maestro status which will let users monitor the progress of large studies (100's of steps) more easily. It was written with codepy in mind in that it groups results by the "run prefix", which corresponds to a "phase" in codepy.
(venv_maestrowf) [crkrenn@rztopaz188 maestrowf]$ maestro status --layout summary $STUDY_PATH
Study: /g/gxx/crkrenn/studies/code
py-study_20220211-183825/codepy-st
udy_20220211-183825_1
┏━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┓
┃ Step Prefix ┃ State ┃ Count ┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━┩
│ phase1 │ FINISHED │ 2 │
│ phase2 │ FINISHED │ 2 │
└─────────────┴──────────┴───────┘
~~With some more work, I can remove some more duplicate code between SummaryStatusRenderer and FlatStatusRenderer~~
The latest version modifies FlatStatusRenderer so that it can be used as a base class for many different status outputs and so that codepy developers can make changes more easily.
I've also added a "narrow2" renderer that, in my mind, is much more useful for many workflows:
(venv_maestrowf) [crkrenn@rztopaz188 maestrowf]$ maestro status --layout flat $STUDY_PATH
Study: /g/g21/crkrenn/studies/codepy-study_20220211-183825/codepy-study_20220211-183825_1
┏━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ ┃ Number ┃
┃ Step Name ┃ Job ID ┃ Workspace ┃ State ┃ Run Time ┃ Elapsed Time ┃ Start Time ┃ Submit Time ┃ End Time ┃ Restarts ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ phase2-samplin │ 29682 │ phase2-samplin │ FINISHED │ 0d:00h:00m:01s │ 0d:00h:00m:01s │ 2022-02-11 │ 2022-02-11 │ 2022-02-11 │ 0 │
│ g2-baseline │ │ g2-baseline │ │ │ │ 18:38:27 │ 18:38:27 │ 18:38:28 │ │
│ phase1-samplin │ 29697 │ phase1-samplin │ FINISHED │ 0d:00h:00m:01s │ 0d:00h:00m:01s │ 2022-02-11 │ 2022-02-11 │ 2022-02-11 │ 0 │
│ g2-baseline │ │ g2-baseline │ │ │ │ 18:38:28 │ 18:38:28 │ 18:38:29 │ │
│ phase2-samplin │ 29709 │ phase2-samplin │ FINISHED │ 0d:00h:00m:02s │ 0d:00h:00m:02s │ 2022-02-11 │ 2022-02-11 │ 2022-02-11 │ 0 │
│ g-baseline │ │ g-baseline │ │ │ │ 18:38:29 │ 18:38:29 │ 18:38:31 │ │
│ phase1-samplin │ 29721 │ phase1-samplin │ FINISHED │ 0d:00h:00m:01s │ 0d:00h:00m:01s │ 2022-02-11 │ 2022-02-11 │ 2022-02-11 │ 0 │
│ g-baseline │ │ g-baseline │ │ │ │ 18:38:31 │ 18:38:31 │ 18:38:32 │ │
└────────────────┴────────┴────────────────┴──────────┴────────────────┴────────────────┴────────────────┴────────────────┴────────────────┴────────────────┘
(venv_maestrowf) [crkrenn@rztopaz188 maestrowf]$ maestro status --layout narrow2 $STUDY_PATH
Study: /g/g21/crkrenn/studies/codepy-study_20220211-183825/codepy-study_20220211-183825_1
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Step Name ┃ Job ID ┃ Workspace ┃ State ┃ Submit Time ┃ Start Time ┃ End Time ┃ Number Restarts ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ phase2-sampling2-baseline │ 29682 │ phase2-sampling2-baseline │ FINISHED │ 2022-02-11 18:38 │ 2022-02-11 18:38 │ 2022-02-11 18:38 │ 0 │
│ phase1-sampling2-baseline │ 29697 │ phase1-sampling2-baseline │ FINISHED │ 2022-02-11 18:38 │ 2022-02-11 18:38 │ 2022-02-11 18:38 │ 0 │
│ phase2-sampling-baseline │ 29709 │ phase2-sampling-baseline │ FINISHED │ 2022-02-11 18:38 │ 2022-02-11 18:38 │ 2022-02-11 18:38 │ 0 │
│ phase1-sampling-baseline │ 29721 │ phase1-sampling-baseline │ FINISHED │ 2022-02-11 18:38 │ 2022-02-11 18:38 │ 2022-02-11 18:38 │ 0 │
└───────────────────────────┴────────┴───────────────────────────┴──────────┴──────────────────┴──────────────────┴──────────────────┴─────────────────┘
Please let me know if you have any questions.
Thanks!
-Chris
@crkrenn -- Thanks for the PR! I'll review this soon and let you know if I have any feedback.
It looks like the style check isn't working, but I also encountered style errors:
> flake8 .
./maestrowf/__init__.py:296:80: E501 line too long (88 > 79 characters)
./maestrowf/__init__.py:314:25: W291 trailing whitespace
./maestrowf/__init__.py:315:22: W291 trailing whitespace
./maestrowf/__init__.py:316:25: W291 trailing whitespace
./maestrowf/__init__.py:317:21: W291 trailing whitespace
./maestrowf/__init__.py:318:27: W291 trailing whitespace
./maestrowf/__init__.py:319:26: W291 trailing whitespace
./maestrowf/__init__.py:320:24: W291 trailing whitespace
./maestrowf/__init__.py:327:27: W291 trailing whitespace
./maestrowf/__init__.py:328:26: W291 trailing whitespace
./maestrowf/__init__.py:329:24: W291 trailing whitespace
@FrankD412, (cc: @jsemler @daub1 @ajkunen @antimatterhorn )
We have been hammering on maestro this week during a forensics exercise. We definitely would benefit from more flexibility in status reports.
I've made the changes you requested (except the color... I use white backgrounds, so I'm happy if you propose a color you like).
I don't know how to fix the readthedocs build error ("ModuleNotFoundError: No module named 'maestrowf'"). Can you help?
Thanks!
-Chris
@crkrenn , this looks a lot like the RTD bugs we patched in June. Try rebasing on the current develop to pull those in and it should fix it (#399 is the one that should fix that CI error).
@jwhite242, Thanks for the suggestion! The readthedocs test is passing now too.
@crkrenn -- I just looked through the changes again and it looks like you addressed everything. I just approved but feel free to continue to add commits if this isn't done.
@FrankD412 (cc: @jwhite242), This is Chris, writing from Cody's account. We have no more planned changes to this pull request. Please merge into develop when you can. Thanks! -Chris
