p4c
p4c copied to clipboard
p4lang
Speed up STF tests by only compiling P4 program once for all test packets
One possible way to address https://github.com/p4lang/p4c/issues/4571
The .p4.test script for an STF test currently does this:
(1) run p4testgen to create multiple STF tests, one packet per file (2) for each such STF test, compile the P4 program and run BMv2 on it
These changes modify step (2) to be:
(2a) for the first STF test, compile the P4 program, copy the BMv2 JSON file to a place outside the tmpXXXXX directory provided on the run-bmv2-test.py command line as a new option, and run BMv2 on it (2b) for the second and later STF tests, copy the BMv2 JSON file from the place outside the tmpXXXXX directory provided ont eh run-bmv2-test.py command line as a new option, skip running the P4 compiler because the BMv2 JSON file is already present, and run BMv2 on it
Basically we avoid running p4c-bm2-ss on the 2nd and later test packets. For some of the larger P4 test programs, they typically generate the maximum of 10 test packets, and today take 4-5 minutes on a 5-year-old x86_64 laptop I use. With these changes, that same test takes about 2 minutes. I think there are about 512 STF tests in the entire test suite, but most of them are shorter than that today, but I would not be surprised if this saved 30 minutes or so across all of them.
@fruffy Is this something you think is worth merging as written? Or would you prefer that your suggested --stf-files [FILES] technique were implemented instead, and abandon the approach in this PR?
@fruffy Is this something you think is worth merging as written? Or would you prefer that your suggested
--stf-files [FILES]technique were implemented instead, and abandon the approach in this PR?
I have not been able to work on the other approach, so I can not say how involved it is. Although I think it requires only few changes. If the other approach is simple enough I'd prefer it because it requires fewer changes on the P4Testgen script side. Until then we can keep this open.
Working on a rewrite of the run-bmv2-stf.py script, which has bitrotted a fair bit. After that rewrite I intend to extend the script to support running multiple files at once.
This PR can either be closed in favor of #4981 or needs to be updated to add the option to the new script.
Closing this PR, in favor of other changes made in test scripts that hopefully achieve a similar gain in efficiency.