openlane2 icon indicating copy to clipboard operation
openlane2 copied to clipboard

Fuzzing Yosys inputs may lead to better results

Open donn opened this issue 1 year ago • 2 comments

Description

Investigation in https://github.com/The-OpenROAD-Project/OpenLane/issues/1697 shows that no-ops in Verilog, such as whitespace and comment, actually have an effect in Yosys.

Per YosysHQ, this is a quasi-feature as it allows us to kinda sorta pick a different "seed" for ABC. @oharboe notes that minor changes to the input can drastically affect the PPA of the netlist.

Yosys for their part provide a command to exploit this: https://github.com/YosysHQ/yosys/issues/3713#issuecomment-1485639838

Proposal

We can use either this command or automate fuzzing of the Verilog source files somehow (maybe by just adding a nondescript amount of line breaks.)

donn avatar Jul 04 '24 11:07 donn

I think at one point or another, it's probably better to actively embrace the chaotic effect of Yosys than try to fight it; I think OpenLane is well-suited to such an idea, because different seeds can be qualified according to the output of post-synthesis STA and area.

Ravenslofty avatar Jul 04 '24 11:07 Ravenslofty

I don't see a conflict between embracing the idea of a stocastic process and also having a deterministic result based upon canonically identical inputs with an explicit fuzzing feature(seed?). This is a common way to handle this situation in many tools: from synthesis to EDA backend tools(like OpenROAD) to geoscience sotcastic history matching.

It is highly advantagous from development point of view to have identical result from a tool when the input is identical(canonically identical is a step further) across machines. I have not seen Yosys giving different results on different machines for the same input, so fortuntely not a problem.

I can see an engineering choice has to be done w.r.t. resources. If I get a determinstic behavior from canonically identical input to Yosys+abc, what other features and improvements would I have to forego?

My thinking is that the canonicalization stage that I propose in https://github.com/The-OpenROAD-Project/OpenROAD-flow-scripts/pull/2104 is the best that I can get today and that if I was given a choice of features and bugfixes that I could have in Yosys, I would pick something else than deterministic yosys+abc behavior based upon canonically identical inputs first.

oharboe avatar Jul 04 '24 13:07 oharboe