Silent failures in the CI

[kareefardi]

trafficstars

Description

Design fail in the CI but the job is reporting success

Expected Behavior

Job should report failure.

Environment report

N/A

Reproduction material

https://github.com/The-OpenROAD-Project/OpenLane/actions/runs/5814254185/job/15763654207

Relevant log output

[INFO]: Using configuration in 'designs/ci/usb/ci_test_config.json'...
[INFO]: PDK Root: /usr/local/pdk
[INFO]: Process Design Kit: sky130A
[INFO]: Setting PDKPATH to /usr/local/pdk/sky130A
no files matched glob pattern "/usr/local/pdk/sky130A/libs.ref/sky130_fd_sc_hd/lef/*.lef"
    while executing
"glob "$::env(PDK_ROOT)/$::env(PDK)/libs.ref/$::env(STD_CELL_LIBRARY)/lef/*.lef""
    invoked from within
"set ::env(CELLS_LEF) [glob "$::env(PDK_ROOT)/$::env(PDK)/libs.ref/$::env(STD_CELL_LIBRARY)/lef/*.lef"]"
    (file "/usr/local/pdk/sky130A/libs.tech/openlane/config.tcl" line 27)
    invoked from within
"source $pdk_config"
    (procedure "prep" line 167)
    invoked from within
"prep {*}$args"
    (procedure "run_non_interactive_mode" line 22)
    invoked from within
"run_non_interactive_mode {*}$argv"
    invoked from within
"if { [info exists flags_map(-interactive)] || [info exists flags_map(-it)] } {
    if { [info exists arg_values(-file)] } {
        run_file [file nor..."
    (file "./flow.tcl" line 403)
2023-08-09 21:32 | FAIL   | ./desig... | OpenLane failed to start up:
Exception in thread Thread-1:
Traceback (most recent call last):
  File "run_designs.py", line 324, in run_design
    subprocess.check_call(command)
  File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['./flow.tcl', '-design', './designs/ci/usb', '-tag', 'ci_test_config', '-config_file', '/openlane/designs/ci/usb/ci_test_config.json', '-overwrite', '-run_hooks', '-verbose', '1']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib64/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "run_designs.py", line 352, in run_design
    print(e.stdout.decode("utf8"))
AttributeError: 'NoneType' object has no attribute 'decode'

2023-08-09 21:32 | Exiting thread 0...
2023-08-09 21:32 | Getting top results...
2023-08-09 21:32 | Benchmarking...
2023-08-09 21:32 | Done.
[Repo Object] Initializing repo Openlane with URL https://github.com/The-OpenROAD-Project/OpenLane.git…
Running on designs ci_test using 1 threads…
Running docker run -v /home/runner/work/OpenLane/OpenLane:/openlane -v /home/runner/work/OpenLane/OpenLane/designs:/openlane/install -v /usr/local/pdk:/usr/local/pdk -u 1001:122 -e PDK_ROOT=/usr/local/pdk -e PDK=sky130A openlane:intermediate-amd64 bash -c 'python3 run_designs.py --disable_timestamp --tag ci_test --threads 1 --benchmark regression_results/benchmark_results/sky130A/sky130_fd_sc_hd.csv --show_output --config_file config ./designs/ci/usb' in /home/runner/work/OpenLane/OpenLane…
Tarballing run...
Created ./reproducible.tar.gz.
Verbose differences within the benchmark:
Passed