NREL
summarize_failures not working because of change in output file formatting
Description:
The format of the openstudio_oputput.log file generated for each model run has changed, which means that the summarize_failures utility no longer works correctly. https://github.com/NREL/ComStock/pull/93 was the first step in addressing because it fixed the filename, but it doesn't capture the step_errors, which is where many of the issues are actually reported.
In the past, it would look like this, and the errors were easy to parse out:
INFO: blah blah
ERROR: blah blah
Now, it is much messier, and the errors are hidden inside step_errors sections of some JSON-formatted sections
INFO: Environment variable SINGULARITYENV_BUILDSTOCKBATCH_VERSION is set, but APPTAINERENV_BUILDSTOCKBATCH_VERSION is preferred
+ ln -s /lib /var/simdata/openstudio/lib
+ ln -s /measures /var/simdata/openstudio/measures
+ ln -s /weather /var/simdata/openstudio/weather
+ openstudio --bundle /var/oscli/Gemfile --bundle_path /var/oscli/gems --bundle_without native_ext run -w in.osw --debug
WorkflowRunOptions:
osw_path=in.osw
no_simulation=false
show_stdout=false
add_timings=false
style_stdout=false
socket_port=0
runOptions={
"debug" : true
}
workflowJSON={
"created_at" : "2024-02-07T08:17:30.641109",
"id" : "bldg0005379up00",
"measure_paths" :
[
"measures"
],
"run_options" :
{
"debug" : true
Suggested solution:
Change this hpc.py code to preferentially read the out.osw file (which is JSON), find all the step_errors sections, and get the errors from there first. If the out.osw file doesn't exist, which can happen if a model has a failure in buildstockbatch, then fall back to parsing the openstudio_output.log file.
