dolphin
dolphin copied to clipboard
Logging improvements
There are some additions/fixes we can do to the current logging setup. Opening this now while waiting to get more specific requirements on what PGE might need
- @gmgunter pointed out that the "Max memory usage" doesn't make it to the log file, if you pass one. that's a bug
- There's work to check that this isn't happening in other spots. The initial file logging was set up quickly and not tested much
It's also worth checking for any places where I might be forcing the output of logs in the "library" portion of dolphin (as opposed to the "application" portion, i.e. dolphin.workflows). The python logging recommendations are to only attach a NullHandler and let applications decide how they want your logs.
possible useful model: https://stackoverflow.com/questions/19425736/how-to-redirect-stdout-and-stderr-to-logger-in-python
FYI, @bhawkins-jpl has a Python logger here that satisfies the NISAR log format spec:
https://github.com/isce-framework/isce3/blob/e4a24ce8927388062825ab8cad95fd88fe937edb/python/packages/nisar/workflows/focus.py#L1184-L1198
Not sure how similar our requirements for logging in OPERA are.
Closing based on JSONL logging in #330 #358