pylint-dev
Update OSS-Fuzz integration
Description
An OSS-Fuzz integration was set up for astroid last year. The build has been failing for a few months and it seems to lack active maintenance.
Would you mind if I fixed the build and assigned myself as a co-maintainer? I maintain the librsvg integration and I am familiar with the OSS-Fuzz platform.
Having a functioning OSS-Fuzz integration could help catch regressions on the main branch before releases are tagged.
Action items
- Fix the build by pinning the version to astroid 3.2.4 for now
- The OSS-Fuzz images are currently stuck on Python 3.8, but there is work being done to support 3.10
- Fixing the build now will allow the corpus to grow, which will benefit future runs against
main(once Python 3.10 support lands)
- Add myself to the list of people who are automatically CC'd on astroid bug reports
Considerations
- Are any of the current maintainers interested in being set as the primary contact for the OSS-Fuzz integration?
- To get full access to the system, you will need a Google/Gmail account
- Note: The associated email address will be listed in the OSS-Fuzz repo without any obfuscation (example)
- Other maintainers can also be automatically CC'd, but there can only be one primary
- To get full access to the system, you will need a Google/Gmail account
- OSS-Fuzz can generate a decent amount of bug reports
- I can help triage the issues, but some maintainers do not want to get notifications from a totally separate system
- Updating the astroid project files on OSS-Fuzz requires the signing of a Google CLA
- I have already signed the CLA, so I can help with PRs if you are uncomfortable with that process
- Google offers monetary rewards for improving the code coverage of existing integrations. I am interested in fixing the build and helping maintain the integration independently of that, but I also have ideas for increasing coverage that might qualify for said rewards.
Google has documentation for the OSS-Fuzz system, but I can also help answer any questions. Thanks!
Thanks for this @correctmost, I'll ask around and get an answer on the primary contact, but I anticipate that the answers to your other questions will be "for sure".
I got a response from @Pierre-Sassoulas that it's okay to list me (@jacobtylerwalls) as primary: jacobtylerwalls [ at ] gmail.com
Would you mind if I fixed the build and assigned myself as a co-maintainer?
Yes, go ahead, and thanks for the help!
Awesome, thanks! I submitted a PR to fix the build and update the maintainers list:
- https://github.com/google/oss-fuzz/pull/12386
The OSS-Fuzz PR was merged, so you should now have access to the following items with your Google account:
- The OSS-Fuzz dashboard
- An overview page of all crashes and hangs/timeouts
- Existing bug reports that were filed automatically for the items on the overview page
If you don't have access yet, it may take a day or two for everything to sync.
We also got CC'd on ~15 existing bug reports. Here's my plan for those:
- Let the automated reproduction tasks run tomorrow with a fixed build
- The build was not fixed for today's run (8-21) because the PR got merged after the run
- Triage any reports that are still marked as reproducible to see if the bug still exists on
main
This is my overall plan for the integration:
- Verify access to the system
- Triage all existing reports
- Submit developer docs to the astroid (or Pylint) repo
- I wrote an OSS-Fuzz developer guide for librsvg that I will adapt
- Increase coverage after a steady state has been reached with the triage efforts
Let me know if you encounter any issues!
Updates for the above tasks:
Verify access to the system
Hopefully you have access now :)
Triage all existing reports
All issues have been triaged. Here's the full list of bugs:
- https://github.com/pylint-dev/astroid/issues/2513
- https://github.com/pylint-dev/astroid/issues/2517
- https://github.com/pylint-dev/astroid/issues/2518
- https://github.com/pylint-dev/astroid/issues/2519
- https://github.com/pylint-dev/astroid/issues/2520
- https://github.com/pylint-dev/astroid/issues/2521
- https://github.com/pylint-dev/astroid/issues/2522
- https://github.com/pylint-dev/astroid/issues/2523
- https://github.com/pylint-dev/astroid/issues/2524
- https://github.com/pylint-dev/astroid/issues/2525
- https://github.com/pylint-dev/astroid/issues/2526
- https://github.com/pylint-dev/astroid/issues/2527
Submit developer docs to the astroid (or Pylint) repo
PR submitted: https://github.com/pylint-dev/pylint/pull/9896
Increase coverage after a steady state has been reached with the triage efforts
Coverage builds are currently broken. I am trying to get more debug info with this PR: https://github.com/google/oss-fuzz/pull/12502.
I will revisit this item after the coverage builds are fixed.
Coverage builds are currently broken. I am trying to get more debug info with this PR: google/oss-fuzz#12502.
Coverage builds were fixed by these changes:
- https://github.com/pylint-dev/astroid/pull/2596
- https://github.com/pylint-dev/astroid/pull/2597
- https://github.com/google/oss-fuzz/pull/12558
I submitted an additional PR to try to increase coverage, which is currently at 66%:
- https://github.com/google/oss-fuzz/pull/12563
Remaining work:
- Triage any new issues that get uncovered by the additional coverage
- Try to upgrade past Python 3.8 on OSS-Fuzz
- https://github.com/google/oss-fuzz/pull/12027 is still pending
- It might also be possible to manually upgrade the astroid project to 3.9, which would allow us to fuzz against
mainagain
OSS-Fuzz was upgraded to Python 3.10.14 yesterday.
I submitted a PR to start using the main branch for fuzzing once again:
- https://github.com/google/oss-fuzz/pull/12763
Once that change takes effect, we can consider the integration up-to-date :).
OSS-Fuzz was recently updated to Python 3.11.13, which seems to be causing many tickets to be incorrectly closed as fixed on the OSS-Fuzz issue tracker.
I assume the crashes will eventually get refiled and will triage any duplicates as they come in.