runbot
runbot copied to clipboard
odoo
Test / use semgrep for security CI
Currently this uses regex which can have a fair bit of false positives for some patterns, unless they are created with extremely complicated patterns.
Performing "semantic" pattern matching is basically the point of semgrep, so it can do thing like relate functions to modules, match function calls except when they're using literal strings (even when the literal string is first set on a local IIRC), etc... so it provides much more precise matching e.g. "execute() except when called on a literal string or an sql.SQL" is relatively easy to express.
The drawback is that semgrep is a bit expensive to run: running the two files I'm attaching takes 52 seconds wallclock, of which 172.40s user and 53.92s system.
The two files I'm attaching are the conversion of the existing patterns to semgrep rules, except for two which maybe should probably remain as regex. The files are encrypted with age/rage using your github ssh keys (I don't think the patterns are super secret as they're pretty easy to infer from the CI / runbot reports but...) @Xavier-Do @d-fence @odony @mart-e
It's called "tar.gz" because github still has not whitelisted age files, so I have to lie about the extension for the file to upload.
The drawback is that semgrep is a bit expensive to run: running the two files I'm attaching takes 52 seconds wallclock, of which 172.40s user and 53.92s system.
In comparison, how long does it takes for the current ones? <1sec? 1min processing could quickly be an issue with 1000 devs pushing all over the place.
In comparison, how long does it takes for the current ones? <1sec?
About 10 seconds according to e.g. https://runbot.odoo.com/runbot/build/9639449 though I think that includes the runbot overhead in spinning up the job.
1min processing could quickly be an issue with 1000 devs pushing all over the place.
OTOH that's to contrast with a community CI build taking about 100mn of CPU (at_install is ~20mn, post_install adds up to ~75mn, and populate low is ~5).