openlane2
openlane2 copied to clipboard
efabless
Support `gzip`ed Liberty (`.lib`) files
Description
Liberty files are huge, which is not ideal when multiple PDKs are installed on a drive. If OL2 supported zipped .lib files naturally, that would be a big thing and super helpful. Plus, should not be super difficult, as OpenROAD seems to support zipped Liberty files already.
Proposal
Either unzip needed file in /tmp on request, or just make the read routines zip-capable.
Happy New Year to you too-- where exactly though because I know you can just put in .lib.gz in most places and the flow would work
Here's the result of a quick test. gzip'ed all .lib, modified the config.tcl accordingly, and ran the smoke-test:
/foss/designs > openlane --smoke-test
[13:45:49] INFO Starting a new run of the 'Classic' flow with the tag 'RUN_2025-01-01_13-45-49'. flow.py:628
[13:45:49] INFO Starting… sequential.py:278
───────────────────────────────────────────────────────────────────────────────── Verilator Lint ──────────────────────────────────────────────────────────────────────────────────
[13:45:49] VERBOSE Running 'Verilator.Lint' at '../../tmp/tmpwltsza4eopenlane2/smoke_test_design/runs/RUN_2025-01-01_13-45-49/01-verilator-lint'… step.py:1122
[13:45:50] VERBOSE Logging subprocess to '../../tmp/tmpwltsza4eopenlane2/smoke_test_design/runs/RUN_2025-01-01_13-45-49/01-verilator-lint/verilator-lint.log'… step.py:1318
- V e r i l a t i o n R e p o r t: Verilator 5.030 2024-10-27 rev v5.030
- Verilator: Built from 0.368 MB sources in 444 modules, into 0.099 MB in 5 C++ files needing 0.000 MB
- Verilator: Walltime 0.032 s (elab=0.005, cvt=0.007, bld=0.000); cpu 0.022 s on 1 threads; alloced 31.770 MB
─────────────────────────────────────────────────────────────────────────── Lint Timing Errors Checker ────────────────────────────────────────────────────────────────────────────
[13:45:50] VERBOSE Running 'Checker.LintTimingConstructs' at step.py:1122
'../../tmp/tmpwltsza4eopenlane2/smoke_test_design/runs/RUN_2025-01-01_13-45-49/02-checker-linttimingconstructs'…
[13:45:50] INFO Check for Lint Timing Errors clear. checker.py:404
─────────────────────────────────────────────────────────────────────────────── Lint Errors Checker ───────────────────────────────────────────────────────────────────────────────
[13:45:50] VERBOSE Running 'Checker.LintErrors' at '../../tmp/tmpwltsza4eopenlane2/smoke_test_design/runs/RUN_2025-01-01_13-45-49/03-checker-linterrors'… step.py:1122
[13:45:50] INFO Check for Lint errors clear. checker.py:132
────────────────────────────────────────────────────────────────────────────── Lint Warnings Checker ──────────────────────────────────────────────────────────────────────────────
[13:45:50] VERBOSE Running 'Checker.LintWarnings' at '../../tmp/tmpwltsza4eopenlane2/smoke_test_design/runs/RUN_2025-01-01_13-45-49/04-checker-lintwarnings'… step.py:1122
[13:45:50] INFO Check for Lint warnings clear. checker.py:132
────────────────────────────────────────────────────────────────────────────── Generate JSON Header ───────────────────────────────────────────────────────────────────────────────
[13:45:50] VERBOSE Running 'Yosys.JsonHeader' at '../../tmp/tmpwltsza4eopenlane2/smoke_test_design/runs/RUN_2025-01-01_13-45-49/05-yosys-jsonheader'… step.py:1122
Classic - Stage 5 - Generate JSON Header ━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4/78 0:00:00
Traceback (most recent call last):
File "/usr/local/bin/openlane", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/openlane/flows/cli.py", line 442, in pdk_resolve_wrapper
return f(*args, pdk_root=pdk_root, pdk=pdk, scl=scl, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/openlane/__main__.py", line 437, in cli
run_included_example(ctx, smoke_test, example, **run_kwargs)
File "/usr/local/lib/python3.10/dist-packages/openlane/__main__.py", line 266, in run_included_example
run(
File "/usr/local/lib/python3.10/dist-packages/openlane/__main__.py", line 147, in run
state_out = flow.start(
File "/usr/local/lib/python3.10/dist-packages/openlane/flows/flow.py", line 657, in start
final_state, step_objects = self.run(
File "/usr/local/lib/python3.10/dist-packages/openlane/flows/sequential.py", line 322, in run
current_state = step.start(
File "/usr/local/lib/python3.10/dist-packages/openlane/steps/step.py", line 1150, in start
views_updates, metrics_updates = self.run(state_in_result, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/openlane/steps/pyosys.py", line 320, in run
views_updates, metrics_updates = super().run(state_in, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/openlane/steps/pyosys.py", line 222, in run
cmd = self.get_command(state_in)
File "/usr/local/lib/python3.10/dist-packages/openlane/steps/pyosys.py", line 313, in get_command
return super().get_command(state_in) + ["--output", out_file]
File "/usr/local/lib/python3.10/dist-packages/openlane/steps/pyosys.py", line 281, in get_command
libs_synth = self.toolbox.remove_cells_from_lib(
File "/usr/local/lib/python3.10/dist-packages/openlane/common/toolbox.py", line 399, in remove_cells_from_lib
for line in input_lib_stream:
File "/usr/lib/python3.10/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
One step further:
Changing this in toolbox.py looks easy enough:
input_lib_stream = gzip.open(file, mode="rt")
But then Yosys is tripping:
[14:01:35] VERBOSE Running 'Yosys.Synthesis' at '../../tmp/tmp3cq1yop4openlane2/smoke_test_design/runs/RUN_2025-01-01_14-01-35/06-yosys-synthesis'… step.py:1122
[14:01:35] VERBOSE Logging subprocess to '../../tmp/tmp3cq1yop4openlane2/smoke_test_design/runs/RUN_2025-01-01_14-01-35/06-yosys-synthesis/yosys-synthesis.log'… step.py:1318
[ERROR] Black-box model '/foss/pdks/sky130A/libs.ref/sky130_fd_sc_hd/lib/sky130_fd_sc_hd__tt_025C_1v80.lib.gz' has an unrecognized file extension.
/----------------------------------------------------------------------------\
| yosys -- Yosys Open SYnthesis Suite |
| Copyright (C) 2012 - 2024 Claire Xenia Wolf <[email protected]> |
| Distributed under an ISC-like license, type "license" to see terms |
\----------------------------------------------------------------------------/
Yosys 0.48 (git sha1 aaa534749, g++ 11.4.0-1ubuntu1~22.04 -fPIC -O3)
[14:01:36] ERROR Subprocess had a non-zero exit. step.py:1364
[14:01:36] ERROR Last 8 line(s): step.py:1369
[ERROR] Black-box model '/foss/pdks/sky130A/libs.ref/sky130_fd_sc_hd/lib/sky130_fd_sc_hd__tt_025C_1v80.lib.gz' has an unrecognized file extension.
/----------------------------------------------------------------------------\
| yosys -- Yosys Open SYnthesis Suite |
| Copyright (C) 2012 - 2024 Claire Xenia Wolf <[email protected]> |
| Distributed under an ISC-like license, type "license" to see terms |
\----------------------------------------------------------------------------/
Yosys 0.48 (git sha1 aaa534749, g++ 11.4.0-1ubuntu1~22.04 -fPIC -O3)
[14:01:36] ERROR Full log file: '../../tmp/tmp3cq1yop4openlane2/smoke_test_design/runs/RUN_2025-01-01_14-01-35/06-yosys-synthesis/yosys-synthesis.log' step.py:1372
Classic - Stage 6 - Synthesis ━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5/78 0:00:00
[14:01:36] ERROR The following error was encountered while running the flow: __main__.py:163
Synthesis: subprocess (255, ['yosys', '-y', '/usr/local/lib/python3.10/dist-packages/openlane/scripts/pyosys/synthesize.py', '--',
'--config-in', '/tmp/tmp3cq1yop4openlane2/smoke_test_design/runs/RUN_2025-01-01_14-01-35/06-yosys-synthesis/config.json', '--extra-in',
'/tmp/tmp3cq1yop4openlane2/smoke_test_design/runs/RUN_2025-01-01_14-01-35/06-yosys-synthesis/extra.json', '--output',
'/tmp/tmp3cq1yop4openlane2/smoke_test_design/runs/RUN_2025-01-01_14-01-35/06-yosys-synthesis/spm.nl.v']) failed
[14:01:36] ERROR OpenLane will now quit.
It looks like Yosys is already supporting gzip'ed Verilog files, so I opened a ticket to support zipped Liberty files as well.
https://github.com/YosysHQ/yosys/issues/4830
I will treat this is a bug so I can push it as 2.3.2. In general we do want to support gzipped files wherever possible.
While I'm there, I'll look into your httpx issue as well.
So as it turns out…
With the Linux Standard Base gzopen, if you pass an uncompressed file for reading, it still opens it, no biggie.
Python's gzip.open, however, will fail upon the first read.
It's always something with this language…
We are now one step further, but the flow fails when running the first STA. OpenSTA should support .lib.gz (see https://github.com/The-OpenROAD-Project/OpenROAD/issues/6484#issuecomment-2575737718) so I don't know what is going on. Yosys seems fine BTW. Here the fail log:
Starting…
Running 'Verilator.Lint' at 'runs/RUN_2025-01-07_17-11-37/01-verilator-lint'…
Logging subprocess to [repr.filename]'runs/RUN_2025-01-07_17-11-37/01-verilator-lint/verilator-lint.log'[/repr.filename]…
Running 'Checker.LintTimingConstructs' at 'runs/RUN_2025-01-07_17-11-37/02-checker-linttimingconstructs'…
Check for Lint Timing Errors clear.
Running 'Checker.LintErrors' at 'runs/RUN_2025-01-07_17-11-37/03-checker-linterrors'…
Check for Lint errors clear.
Running 'Checker.LintWarnings' at 'runs/RUN_2025-01-07_17-11-37/04-checker-lintwarnings'…
Check for Lint warnings clear.
Running 'Yosys.JsonHeader' at 'runs/RUN_2025-01-07_17-11-37/05-yosys-jsonheader'…
Logging subprocess to [repr.filename]'runs/RUN_2025-01-07_17-11-37/05-yosys-jsonheader/yosys-jsonheader.log'[/repr.filename]…
Running 'Yosys.Synthesis' at 'runs/RUN_2025-01-07_17-11-37/06-yosys-synthesis'…
Logging subprocess to [repr.filename]'runs/RUN_2025-01-07_17-11-37/06-yosys-synthesis/yosys-synthesis.log'[/repr.filename]…
Parsing synthesis checks…
Running 'Checker.YosysUnmappedCells' at 'runs/RUN_2025-01-07_17-11-37/07-checker-yosysunmappedcells'…
Check for Unmapped Yosys instances clear.
Running 'Checker.YosysSynthChecks' at 'runs/RUN_2025-01-07_17-11-37/08-checker-yosyssynthchecks'…
Check for Yosys check errors clear.
Running 'Checker.NetlistAssignStatements' at 'runs/RUN_2025-01-07_17-11-37/09-checker-netlistassignstatements'…
Running 'OpenROAD.CheckSDCFiles' at 'runs/RUN_2025-01-07_17-11-37/10-openroad-checksdcfiles'…
'PNR_SDC_FILE' is not defined. Using generic fallback SDC for OpenROAD PnR steps.
'SIGNOFF_SDC_FILE' is not defined. Using generic fallback SDC for OpenROAD PnR steps.
Running 'OpenROAD.CheckMacroInstances' at 'runs/RUN_2025-01-07_17-11-37/11-openroad-checkmacroinstances'…
No macros found, skipping instance check…
Running 'OpenROAD.STAPrePNR' at 'runs/RUN_2025-01-07_17-11-37/12-openroad-staprepnr'…
No SCL lib files found for nom_tt_025C_1v80.
Starting STA for the nom_tt_025C_1v80 timing corner…
No SCL lib files found for nom_ss_100C_1v60.
Skipping corner nom_ss_100C_1v60 for STA (identical to nom_tt_025C_1v80 at this stage)…
No SCL lib files found for nom_ff_n40C_1v95.
Skipping corner nom_ff_n40C_1v95 for STA (identical to nom_tt_025C_1v80 at this stage)…
No SCL lib files found for min_tt_025C_1v80.
Skipping corner min_tt_025C_1v80 for STA (identical to nom_tt_025C_1v80 at this stage)…
Logging subprocess to [repr.filename]'runs/RUN_2025-01-07_17-11-37/12-openroad-staprepnr/nom_tt_025C_1v80/sta.log'[/repr.filename]…
No SCL lib files found for min_ss_100C_1v60.
Skipping corner min_ss_100C_1v60 for STA (identical to nom_tt_025C_1v80 at this stage)…
No SCL lib files found for min_ff_n40C_1v95.
Skipping corner min_ff_n40C_1v95 for STA (identical to nom_tt_025C_1v80 at this stage)…
No SCL lib files found for max_tt_025C_1v80.
Skipping corner max_tt_025C_1v80 for STA (identical to nom_tt_025C_1v80 at this stage)…
No SCL lib files found for max_ss_100C_1v60.
Skipping corner max_ss_100C_1v60 for STA (identical to nom_tt_025C_1v80 at this stage)…
No SCL lib files found for max_ff_n40C_1v95.
Skipping corner max_ff_n40C_1v95 for STA (identical to nom_tt_025C_1v80 at this stage)…
Subprocess had a non-zero exit.
Last 10 line(s):
Warning: /foss/designs/example/dig/runs/RUN_2025-01-07_17-11-37/06-yosys-synthesis/counter.nl.v line 256, module sky130_fd_sc_hd__nor2_2 not found. Creating black box for _131_.
Warning: /foss/designs/example/dig/runs/RUN_2025-01-07_17-11-37/06-yosys-synthesis/counter.nl.v line 281, module sky130_fd_sc_hd__a21oi_2 not found. Creating black box for _136_.
Warning: /foss/designs/example/dig/runs/RUN_2025-01-07_17-11-37/06-yosys-synthesis/counter.nl.v line 287, module sky130_fd_sc_hd__and3_2 not found. Creating black box for _137_.
Warning: /foss/designs/example/dig/runs/RUN_2025-01-07_17-11-37/06-yosys-synthesis/counter.nl.v line 344, module sky130_fd_sc_hd__a31oi_2 not found. Creating black box for _147_.
Warning: /foss/designs/example/dig/runs/RUN_2025-01-07_17-11-37/06-yosys-synthesis/counter.nl.v line 668, module sky130_fd_sc_hd__dfrtp_2 not found. Creating black box for _212_.
Reading design constraints file at '/usr/local/lib/python3.12/dist-packages/openlane/scripts/base.sdc'…
[INFO] Using clock i_clk…
[INFO] Setting output delay to: 4
[INFO] Setting input delay to: 4
Error: base.sdc line 54, 'sky130_fd_sc_hd__inv_2' not found.
Full log file: 'runs/RUN_2025-01-07_17-11-37/12-openroad-staprepnr/nom_tt_025C_1v80/sta.log'
Failed STA for the nom_tt_025C_1v80 timing corner
@donn Maybe you know where the fail happens, whether OL2 or OpenSTA related. If OpenSTA, then I can open a ticket there, but in the moment I don't know what is going on.