PDNGen issue in for a SOC with Mix of hard-macro with met3/met4 and met4/met5 power mesh
Description
My SOC project has Multiple Hardmarco and all these hardmacro had power hook up in met-3/met-4. Recently i have added a new macro which had power hook-up of met-4/met-5. Openlane PDN generation flow does not shown any failure during PDN generation, But when we review the design, none of the hardmacro power is connected.
Here is working case power hook-up:

Here is Non-working case power hook-up:

Environment
This issue is observed with latest MPW-6 openlane flow; But PDN flows does not fails , No not clear on how to share the complete openlane setup here. My Git work area: https://github.com/dineshannayya/riscduino
I have created a PDN standalone setup to re-produce the issue.
Reproduction Material
Attached the Non working case:
riscduino_pdn_non_working.tar.gz
Attached the working case: riscduino_pdn_working.tar.gz
Run command: run_cmd
To check the failure review the output/user_project_wrapper.def
Expected behavior
PDN flow should fails to bring the user attention and also if there is way to handle these type of macro integration through Openlane flow.
Logs
Additional info: In the failure setup, when i deleted the digital_pll definition in the .def file. I see PDN flow works fine and power-hook connected to all the hardmarco.
@dineshannayya
Can you share me pdn_working case GLB_RT_OBS?
Is this one ?
set ::env(GLB_RT_OBS) "
li1 150 130 833.1 546.54,
met1 150 130 833.1 546.54,
met2 150 130 833.1 546.54,
met3 150 130 833.1 546.54,\
li1 950 130 1633.1 546.54,\
met1 950 130 1633.1 546.54,\
met2 950 130 1633.1 546.54,\
met3 950 130 1633.1 546.54,\
li1 150 750 833.1 1166.54,\
met1 150 750 833.1 1166.54,\
met2 150 750 833.1 1166.54,\
met3 150 750 833.1 1166.54,\
met1 950 650 1760 660 ,\
met3 950 650 1760 660 ,\
met5 0 0 2920 3520"
As digital_pll macro used met5 layer is the different from both testcase lef files.
As per maco hardening met5 not allowed to use. plz check link here: https://github.com/The-OpenROAD-Project/OpenLane/blob/master/docs/source/hardening_macros.md#power-grid-pdn
Re-harden the digital_pll macro and let us know if same issue repeats.
This digital_pll is hardmarco picked from caravel project. If caravel/efabless team able to integrate this macro why it's issue for my project ? In MPW-5 time I have integrated DFFRAM from caravel which also had power hook in met4/met5 and I was able work with MPW-5 tool set which has DRC issue which got fixed with MACRO location movement. Is this new requirement for MPW-6 ?
Ok. I will try to clone your repo and check it.
Here is working case with MPW-5 tool with same MPW-6 DEF and LEF included with digital_pll

Attached my MPW-5 PDN setup riscduino_working_mpw5.tar.gz
@maliberty fyi
@dineshannayya
Can you give make survey || python3 ./env.py issue-survey
python3 ./env.py issue-survey Kernel: Linux v5.13.0-44-generic Distribution: ubuntu 20.04 Python: v3.8.10 (OK) Container Engine: docker v20.10.16 (OK) OpenLane Git Version: 0dc6fb79c91082b94f8ded78d70f8bacbab96bf2 pip: INSTALLED pip:venv: INSTALLED
PDK Version Verification Status: OK
Git Log (Last 3 Commits)
0dc6fb7 2022-05-17T08:24:25-07:00 Enable PL_ROUTABILITY_DRIVEN and PL_TIME_DRIVEN (#1092) - Anton Blanchard - (HEAD -> master, tag: 2022.05.18_02.12.32, origin/master, origin/HEAD)
fba1ad0 2022-05-17T14:45:07+02:00 Remove unnecessary calls to read_verilog (#1084) - Anton Blanchard - ()
9d463c0 2022-05-17T14:43:10+02:00 Add FP_IO_UNMATCHED_ERROR flag (#1086) - Kareem Farid - ()
@arlpetergadfort any comments?
@maliberty I'm unable to look at the files until late today. But if they are using the OpenLane flow, I don't know if there is a way to specify multiple macros in it. The default flow just makes a single -default grid for all macros (assuming a single layer for contact). If there are macros with different requirements, they need a grid defined for them.
I don't think there is an option currently:
define_pdn_grid \
-macro \
-default \
-name macro \
-starts_with POWER \
-halo "$::env(FP_PDN_HORIZONTAL_HALO) $::env(FP_PDN_VERTICAL_HALO)"
add_pdn_connect \
-grid macro \
-layers "$::env(FP_PDN_LOWER_LAYER) $::env(FP_PDN_UPPER_LAYER)"
is all there is.
In that case, the current setup will not be able to connect them. OL will need to be augmented to support multiple different definitions of macros (or the macro will need to have additional layers of metal inserted to comply with all the macros in the design.)
If we see the power mesh inside the digital pll , Vertical => MET4 and Horizontal => MET5
There should not be any issue in Golbal Horizontal PDN Power Strip MPW5 to connect the Macro Vertical MET4. This was the case until the MPW-5 openlane tool set. I used to face DRC issue due to over-lapping Global Horizontal Power Strip MET5 with Macro Horizontal MET5. By moving the Hardmacro relative position I used to clear the DRC issue.
.
If you see the current issue, PDN hook-up is missed for even macro with MET3/MET4 power mesh and I don't see any PDN error reporting during this case
@arlpetergadfort
Please check digital_pll macro lef. While integrating causing the issue.
Additional info Shared by @dineshannayya : In the failure setup, when i deleted the digital_pll definition in the .def file. I see PDN flow works fine and power-hook connected to all the hardmarco.
@dineshannayya with latest openroad commit non-working log as follows:
[WARNING PSM-0038] Unconnected PDN node on net vccd1 at location (-8.480um, 1910.880um), layer: 6.
[WARNING PSM-0038] Unconnected PDN node on net vccd1 at location (-8.480um, 1820.880um), layer: 6.
[WARNING PSM-0038] Unconnected PDN node on net vccd1 at location (-8.480um, 1730.880um), layer: 6.
[WARNING PSM-0038] Unconnected PDN node on net vccd1 at location (-8.480um, 1640.880um), layer: 6.
[WARNING PSM-0038] Unconnected PDN node on net vccd1 at location (-8.480um, 1550.880um), layer: 6.
[WARNING PSM-0038] Unconnected PDN node on net vccd1 at location (-8.480um, 1460.880um), layer: 6.
[ERROR PSM-0069] Check connectivity failed.
Error: pdn.tcl, 117 PSM-0069
There was connectivity issue.
But other working case its passing fine no error.
[WARNING PSM-0030] VSRC location at (2805.520um, 3090.880um) and size 10.000um, is not located on an existing power stripe node. Moving to closest node at (2810.520um, 3080.880um).
[WARNING PSM-0030] VSRC location at (565.520um, 3230.880um) and size 10.000um, is not located on an existing power stripe node. Moving to closest node at (570.520um, 3260.880um).
[WARNING PSM-0030] VSRC location at (1405.520um, 3230.880um) and size 10.000um, is not located on an existing power stripe node. Moving to closest node at (1370.520um, 3260.880um).
[WARNING PSM-0030] VSRC location at (2245.520um, 3230.880um) and size 10.000um, is not located on an existing power stripe node. Moving to closest node at (2250.520um, 3260.880um).
[INFO PSM-0031] Number of PDN nodes on net vccd1 = 1555.
[INFO PSM-0064] Number of voltage sources = 88.
[INFO PSM-0040] All PDN stripes on net vccd1 are connected.
########## IR report #################
Worstcase voltage: 1.80e+00 V
Average IR drop : 2.01e-13 V
Worstcase IR drop: 3.26e-12 V
######################################
@dineshannayya I found a bug in the instance grid code, see PR: https://github.com/The-OpenROAD-Project/OpenROAD/pull/1997 For the PLL, because it has metal on met5, PDNGEN will not place straps over it, so it will remain disconnected. Looking at the grid in the PLL and the grid PDNGEN is building, it would short vssd1 and vccd1, so my recommendation would be to limit the macros to met4 to PDNGEN can connect at the last level.
@arlpetergadfort As i informed in mail thread, I am re-using this hardmarco available in Efabless/caravel project. As this block has some analog component, I will not be able to re-harden it my side.
Regarding the latest PDNGEN integration issue with Hard macro with MET4/MET5. Previously I was able integrated similar IP in MPW-5 tool set & I used to face DRC issue due to over-lapping at MET5. By moving the Hard macro relative position I used to clear the DRC issue. I have already attached the working MPW-5 example.
I see this PDN issue is new restriction occurred in MPW-6 tool set onward.
Currently I am planning to move ahead with using MPW-5 tools for my integration - As I am able to complete the Openlane flow without LVS issue with MPW-5 tools set.
If the old pdgen routed over the macro it was a bug. The fact that you had to manually align the macro suggests it was so. It is not a supported feature and would be an enhancement request. You could
- Unplace the macro, run pdngen and then replace the macro as you did manually before.
- Strip the top layer power shapes from the macro and allow pdngen to rebuild the connections.
@maliberty You will be able to move this ticket to enhancement request or you want me to raise a separate request.
Just because pdgen need some spaces in met5 for power strip routing/power hook and restricting macro's from using met5 for routing does not look to be ideal solution. I can say old pdngen bug was a nice to have feature (May be with extra option like allow DRC violation). Previously it was one time effort to get the DRC failure location due to Global PDN strip overlap with Macro Power Strip and adjust macro relative position accordingly.
I will cross-check with efabless team on their implementation plan for caravel project as i see they have multiple sub macro (listed below) uses MET5 for power routing caravel macro's using MET5: caravan_power_routing, caravel_clocking, chip_io_alt, chip_io, digital_pll, gpio_control_block,gpio_defaults_block,housekeeping, simple_por, spare_logic_block,user_id_programming
We can consider it as an enhancement request. I know Peter is busy at the moment - how urgent is this for you?
This is not blocking my current work, I am using latest Openlane for Macro hardening and Previous openlane (MPW5) for top-level integration + Power Hook up. This will be creating confusion for any one try to fork my repo for their analysis.
@maliberty Is there is any plan to prioritize fixing this issue. Currently I need to switch between MPW-5 and MPW-7 tool set get clean database
There was no plan to raise the priority since you said it is not blocking your work. Has that changed?
I am trying to integrate ADC macro which has two power domain. Macro with two power domain PDN integration only supported in MPW-7 tool set only. Now i need to hack the digital pll lef to work with MPW-7 tool set. I also notice other project also reporting similar issue while integrating caravel digital pll macro. I appreciate if you can raise priority to have clean flow.
@arlpetergadfort when you are back can you give this some attention? If not I'll find someone else to assign it to.
@dineshannayya this should allow you to connect the macros: https://github.com/The-OpenROAD-Project/OpenROAD/pull/2134 It will require placement to ensure the straps can go through the macro (the placement uploaded doesn't).
@arlpetergadfort I have picked your src/pdn/src/grid.cpp and locally compiled the openroad tool and validated the pdn changes. After moving the digital pll by 5 micro (100 to 105 micron) in Y Direction, I see there is vccd1/vssd1 feed through the digital pll in MET5.
One additional observation is this first time i see MET4 also feed through this block. I have not yet run full openlane flow to see any issue.

There is no OBS blockage in met4 or met5 (so the behavior is going to be the same). If you move it over by 5um in X I think both the met4 straps will avoid the pins in met4 and connect. Or if you don't want met4, you will need to add a routing blockage over the block to prevent this.