With tightly packed macros, LVS fails due to hanging strips of metal on the PDN

[mattvenn]

Description

With tightly packed macros, LVS fails due to hanging strips of metal on the PDN

Environment

Kernel: Linux v5.13.0-40-generic
Distribution: ubuntu 20.04
Python: v3.8.10 (OK)
Container Engine: docker v20.10.7 (OK)
OpenLane Git Version: -dev
pip:click: INSTALLED
pip:pyyaml: INSTALLED
pip:venv: INSTALLED
---
PDK Version Verification Status: OK
---
Git Log (Last 3 Commits)

e4bfdd7 2022-02-22T16:59:14-03:00 Remove pip install, no longer needed (#953) - Vitor Bandeira -  (grafted, HEAD, tag: 2022.02.23_02.50.41)

Reproduction Material

• tarball not easy as this is Caravel - see repo: https://github.com/mattvenn/tapeout_100
• List the commands used to run the design:
• make install (installs caravel submodule)
• make uncompress (uncompresses gds files)
• user_project_wrapper

Expected behavior

LVS shouldn't fail

Logs

final lines of user_project_wrapper/runs/user_project_wrapper/logs/finishing/26-user_project_wrapper.lef.log:

vssd1                                      |(no matching pin)                          
vssd1_uq0                                  |(no matching pin)                          
vssd1_uq1                                  |(no matching pin)                          
vssd1_uq2                                  |(no matching pin)                          
vssd1_uq3                                  |(no matching pin)      

[vijayank88]

@mattvenn I am trying to clone your repo with https://github.com/The-OpenROAD-Project/OpenLane-MPW-CI and running its got failed to find module wrapped_icestudio_test

[INFO]: Preparing LEF files for the max corner...
[INFO]: Looking for files defined in ::env(EXTRA_GDS_FILES)  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_function_generator.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_vga_clock.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_frequency_counter.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_rgb_mixer.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_hack_soc_dffram.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_teras.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_alu74181.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_vgademo_on_fpga.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_silife.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_acorn_prng.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_hsv_mixer.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_skullfet.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wb_bridge_2way.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wb_openram_wrapper.gds  /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/sky130_sram_1kbyte_1rw1r_32x256_8.gds  ...
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_function_generator.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_vga_clock.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_frequency_counter.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_rgb_mixer.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_hack_soc_dffram.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_teras.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_alu74181.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_vgademo_on_fpga.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_silife.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_acorn_prng.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_hsv_mixer.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wrapped_skullfet.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wb_bridge_2way.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/wb_openram_wrapper.gds exists.
[INFO]: /openlane/designs/tapeout_100/openlane/user_project_wrapper/../../gds/sky130_sram_1kbyte_1rw1r_32x256_8.gds exists.
[STEP 1]
[INFO]: Running Synthesis...
[STEP 2]
[INFO]: Running Single-Corner Static Timing Analysis...
[INFO]: Creating a netlist with power/ground pins.
[STEP 3]
[INFO]: Running Initial Floorplanning...
[ERROR]: Floorplanning failed
[ERROR]: module wrapped_icestudio_test not found in /openlane/designs/tapeout_100/runs/RUN_2022.04.30_09.48.18/tmp/merged.nom.lef
[ERROR]: Check whether EXTRA_LEFS is set appropriately
[INFO]: Generating final set of reports...
[INFO]: Created manufacturability report at 'runs/RUN_2022.04.30_09.48.18/reports/manufacturability.rpt'.
[INFO]: Created metrics report at 'runs/RUN_2022.04.30_09.48.18/reports/metrics.csv'.
[INFO]: Saving runtime environment...
[ERROR]: Flow failed.

    while executing
"flow_fail"
    (procedure "check_floorplan_missing_lef" line 11)
    invoked from within
"check_floorplan_missing_lef"
    (procedure "init_floorplan" line 12)
    invoked from within
"init_floorplan"
    (procedure "run_floorplan" line 7)
    invoked from within
"[lindex $step_exe 0] [lindex $step_exe 1] "
    (procedure "run_non_interactive_mode" line 55)
    invoked from within
"run_non_interactive_mode {*}$argv"
    invoked from within
"if { [info exists flags_map(-interactive)] || [info exists flags_map(-it)] } {
	if { [info exists arg_values(-file)] } {
		run_file [file normalize $a..."
    (file "./flow.tcl" line 427)
make: *** [quick_run] Error 1
make: Leaving directory `/home/vijayan/MPW_CI/OpenLane-MPW-CI/OpenLane'

Can you update required module and openlane/user_project_wrapper/config.tcl to run the flow to re-produce your issue at our end?

[maliberty]

Why does it being caravel make it hard to produce a tarball? @donn any thoughts?

[donn]

You can ask the caravel team.

[mattvenn]

@maliberty because the caravel_user_project is 8.2G. So I need to know what to .tar.gz in order to capture what you need for a test case

[mattvenn]

@vijayank88 thanks for taking a look at this. I've fixed the example repo, my bad I should have tested it locally.

[antonblanchard]

I'm not sure it's the best way, but I'll throw an exit 1 into the step that I want to debug, and Openlane should die and populate openroad_issue_reproducible. Assuming things start going wrong at the PDN stage, add it to scripts/openlane/pdn.tcl.

@donn or @maliberty might have a better idea.

[mattvenn]

actually I did know this trick but thought it would only work with openlane, rather than inside caravel_user_project. Thanks Anton!

[vijayank88]

@mattvenn I cloned your repo and its fail with LVS mismach error.

Its similar to #1067

[STEP 29]
[INFO]: Running LEF LVS...
[ERROR]: There are LVS errors in the design: See 'runs/RUN_2022.05.02_12.28.56/logs/signoff/29-user_project_wrapper.lvs.lef.log' for details.
[INFO]: Generating final set of reports...
[INFO]: Created manufacturability report at 'runs/RUN_2022.05.02_12.28.56/reports/manufacturability.rpt'.
[INFO]: Created metrics report at 'runs/RUN_2022.05.02_12.28.56/reports/metrics.csv'.
[INFO]: Saving runtime environment...
[ERROR]: Flow failed.
[INFO]: The failure may have been because of the following warnings:
[WARNING]: Skipping Tap/Decap Insertion.
[WARNING]: Specifying a routing obstruction is now done using the coordinates
[WARNING]: of its bounding box instead of the now deprecated (x, y, size_x, size_y).


    while executing
"flow_fail"
    (procedure "quit_on_lvs_error" line 12)
    invoked from within
"quit_on_lvs_error -log $count_lvs_log"
    (procedure "run_lvs" line 79)
    invoked from within
"run_lvs"
    (procedure "run_lvs_step" line 11)
    invoked from within

[vijayank88]

I'm not sure it's the best way, but I'll throw an exit 1 into the step that I want to debug, and Openlane should die and populate openroad_issue_reproducible. Assuming things start going wrong at the PDN stage, add it to scripts/openlane/pdn.tcl.

@donn or @maliberty might have a better idea.

This way its works and produce openroad_issue_reproducible, but user should remove exit 1 from both script.

[vijayank88]

@maliberty

Fixing #1067 will resolve this error too.

[mattvenn]

I don't think so. I think this is valid that netgen is showing an error as there are many unconnected strips of metal. The problem is that the PDN is generating these hanging strips that don't connect to anything.

[maliberty]

I want to get OL to the new pdngen and see if that resolves the issue rather than fixing the old pdngen. I'm hoping it is close now.

[maliberty]

Can you try this with https://github.com/The-OpenROAD-Project/OpenLane/pull/1059 ?

[vijayank88]

@antonblanchard Here @mattvenn re-used macros wrapped_icestudio_test 7 times.

Refer repo here: https://github.com/mattvenn/tapeout_100/blob/tapeout_100/openlane/user_project_wrapper/macro.cfg

How to define MACRO_HOOKS?

set ::env(FP_PDN_MACRO_HOOKS) "\
	mprj vccd1 vssd1"

[antonblanchard]

@vijayank88 @mattvenn I think you want:

set ::env(FP_PDN_MACRO_HOOKS) "\
	wrapped_icestudio_test_0 vccd1 vssd1, \
	wrapped_icestudio_test_1 vccd1 vssd1, \
	wrapped_icestudio_test_2 vccd1 vssd1, \
	wrapped_icestudio_test_3 vccd1 vssd1, \
	wrapped_icestudio_test_4 vccd1 vssd1, \
	wrapped_icestudio_test_5 vccd1 vssd1, \
	wrapped_icestudio_test_6 vccd1 vssd1, \
	wrapped_icestudio_test_7 vccd1 vssd1"

@maliberty I notice the Openlane documentation claims macros are connected to the first power domain if they are unspecified:

| `FP_PDN_MACRO_HOOKS` | Specifies explicit power connections of internal macros to the top level power grid. Comma separated list of macro instance names and power domain vdd and ground net names: `<instance_name> <vdd_net> <gnd_net>` <br> (Default: macros are connected to the first power domain

That's not correct is it? Should we remove that comment?

[vijayank88]

@mattvenn Follow above FP_PDN_MACRO_HOOKS use set ::env(LVS_CONNECT_BY_LABEL) 1 Able to complete the flow successfully at my end. Please do confirm.

[maliberty]

@antonblanchard re "Should we remove that comment?" Yes. I'm not clear that was actually true before either but I haven't tested it. Arbitrarily connecting macros seems dangerous.

[maliberty]

@vijayank88 there should not be dangling metal. Connecting by label is a good workaround but it masks the issue. Have you verified if the problem still exists with the new pdngen without this workaround? If so please post an updated test case.

[antonblanchard]

@antonblanchard re "Should we remove that comment?" Yes. I'm not clear that was actually true before either but I haven't tested it. Arbitrarily connecting macros seems dangerous.

Created https://github.com/The-OpenROAD-Project/OpenLane/pull/1091

[vijayank88]

@maliberty Latest PDN stage attached here: openroad_issue_reproducible.zip

[maliberty]

@arlpetergadfort there are dangling pieces of met4 between the macros: The right piece has a via but the left does not. The right belongs to the single met5 vdd strap while the left is part of vss and has nothing to connect to.

[maliberty]

[maliberty]

[mattvenn]

@vijayank88 @mattvenn I think you want:

set ::env(FP_PDN_MACRO_HOOKS) "\
	wrapped_icestudio_test_0 vccd1 vssd1, \
	wrapped_icestudio_test_1 vccd1 vssd1, \
	wrapped_icestudio_test_2 vccd1 vssd1, \
	wrapped_icestudio_test_3 vccd1 vssd1, \
	wrapped_icestudio_test_4 vccd1 vssd1, \
	wrapped_icestudio_test_5 vccd1 vssd1, \
	wrapped_icestudio_test_6 vccd1 vssd1, \
	wrapped_icestudio_test_7 vccd1 vssd1"

@maliberty I notice the Openlane documentation claims macros are connected to the first power domain if they are unspecified:

| `FP_PDN_MACRO_HOOKS` | Specifies explicit power connections of internal macros to the top level power grid. Comma separated list of macro instance names and power domain vdd and ground net names: `<instance_name> <vdd_net> <gnd_net>` <br> (Default: macros are connected to the first power domain

That's not correct is it? Should we remove that comment?

I've found through experimentation that the FP_PDN_MACRO_HOOKS isn't required - macros get connected to power anyway. What does it do? I've also tried using it, and I still get warnings about macros not being powered.

[mattvenn]

@vijayank88 I tried your suggestions and got this new error:

[ERROR PDN-0165] Conflict found, instance wrapped_icestudio_test_0 is part of two grid definitions (CORE_macro_grid_2, CORE_macro_grid_1)

[jurevreca12]

@mattvenn Hey matt, did you maybe manage to fix this issue? (Conflict found...) I am getting the same issue.

[mattvenn]

No, haven't found a fix yet.

[jurevreca12]

@vijayank88 I noticed one thing after closing the issue. I commented out the macro set ::env(FP_PDN_MACRO_HOOKS) . If I don't do that i still get the same error as Matt. Is this a problem?

[vijayank88]

@jurevreca12 Open a new issue with clear description and test case.

[jurevreca12]

https://github.com/The-OpenROAD-Project/OpenLane/issues/1104