Support 3D Custom Switch Blocks

[saaramahmoudi]

Description

This pull request support 3d custom switch blocks in the architecture file and automatically add the inter-die edges between tracks in different layer to RR graph.

Types of changes

• [ ] Bug fix (change which fixes an issue)
• [x] New feature (change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• [x] My change requires a change to the documentation
• [ ] I have updated the documentation accordingly
• [ ] I have added tests to cover my changes
• [x] All new and existing tests passed

[vaughnbetz]

Please attach VTR, Titan and Koios QoR data to make sure we haven't degraded it.

[vaughnbetz]

Another possible future work (maybe fits in the same time): bidirectional wires.

[saaramahmoudi]

@vaughnbetz Here is QoR check for VTR and Titan benchmarks for 2D FPGAs. QoR check

[vaughnbetz]

Thanks. Please also do a comparison of maximum memory footprint. There are some CI QoR failures that are due to an increase in maximum memory. nightly_test3 has some for example with a complex switch block (up over 20% vs. golden results).

[saaramahmoudi]

@vaughnbetz Titan23 and VTR show the qor_compare script outputs for these benchmarks. 3D custom SBs have some max memory increase for both, 4% and 2.5%. I can dig into it more if we can't merge this with max memory increase.

[vaughnbetz]

It's worth digging into some. We could merge without a fix as it isn't catastrophic, but LU230 has the highest Titan memory footprint and has the highest increase (about 20 or 25%). So that's noticable, and it's interesting most designs don't change but a few (generally big) do.
To merge now you'll also have to update a bunch of QoR results, so it will take a bit of effort. Can you think of any big data structures allocated in the switch blocks that should be put on a diet? It seems adding 3D switch boxes shouldn't increase memory footprint. Is there a change to the rr-graph data structures (I didn't think there was one) or just a change to the generation code? If only generation code, we should be able to keep any memory increase minimal.

[vaughnbetz]

nightly_test7 failed but is a quality improvement so OK to update golden. Hopefully the others are the same.

regression_tests/vtr_reg_nightly_test7/3d_titan_other...[Fail] 17:35:26 | [Fail] 17:35:26 | 3d_full_OPIN_inter_die_stratixiv_arch.timing.xml/leon3mp_stratixiv_arch_timing.blif/common crit_path_route_time relative value 0.07425381635907953 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 438.9 17:35:26 | 17:35:26 | regression_tests/vtr_reg_nightly_test7/3d_titan_other...[Fail] 17:35:26 | [Fail] 17:35:26 | 3d_full_OPIN_inter_die_stratixiv_arch.timing.xml/smithwaterman_stratixiv_arch_timing.blif/common crit_path_route_time relative value 0.09664310954063604 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 509.4

[vaughnbetz]

Quite a few CI failures ... can you take a look @saaramahmoudi ?

[saaramahmoudi]

Quite a few CI failures ... can you take a look @saaramahmoudi ?

Yes, I will. But before that, we still have some routing problem that might be related to router lookahead. I already talked to @amin1377, and he agreed to take a look into the new commits. After we found all bugs, I will investigate CI issues.

[vaughnbetz]

Please go ahead and merge this after the conflicts are resolved, and a QoR test on Titan to ensure 2D QoR isn't affected. There are two comments above that should go in an issue to resolve afterwards:

• move --custom_3d_sb_fanin_fanout to the architecture file as a new tag, rather than as a command line option. It should go in the section.
• Add a big comment somewhere (maybe in rr_graph.cpp) about how custom_3d_sb_fanin_fanout works. It should detail exactly what the rr graph this generates looks like.

[vaughnbetz]

Issue #2694 is tracking the updates listed above.