google
[infra] upgrade python
/gcbrun trial_build.py all --sanitizer coverage address --fuzzing-engine libfuzzer
Github is breaking this by messing with something bazel (which i really wish we weren't installing in base images) depends on: https://twitter.com/mihaimaruseac/status/1620179105038876672
/gcbrun trial_build.py all --sanitizer coverage address --fuzzing-engine libfuzzer
failures tint spirv-tools pycrypto wheel proto-plus-python http-pattern-matcher airflow
spurious wolfssl cel-cpp wolfmqtt tinyusb tensorflow jersey
/gcbrun trial_build.py tint spirv-tools pycrypto wheel proto-plus-python http-pattern-matcher airflow --sanitizer coverage address --fuzzing-engine libfuzzer
http-pattern-matcher is stable and in maintenance mode, so I am not too worried. Fuzzer hasn't found anything recently
it's really hard to make sense of the failures because of the libxml2 repo breakage and introspector's bad state (breaking everywhere) I'm gonna retry without introspector.
fails: glib https://pantheon.corp.google.com/cloud-build/builds;region=us-central1/9d040cc7-10a3-4bcd-80b0-544423fdc6e4?project=oss-fuzz gdk-pixbuf https://pantheon.corp.google.com/cloud-build/builds;region=us-central1/24d24b26-6f9d-4aaa-a2aa-82fc6f25a944?project=oss-fuzz wheel https://pantheon.corp.google.com/cloud-build/builds;region=us-central1/27eac926-2554-4f1f-a192-2b634fcac6d8?project=oss-fuzz python-rsa https://pantheon.corp.google.com/cloud-build/builds;region=us-central1/8f20b1f8-fcfc-42a5-bdaf-03a7dc53457a;tab=detail?project=oss-fuzz proto-plus-python https://pantheon.corp.google.com/cloud-build/builds;region=us-central1/41313c36-c59f-4042-bd8b-d15554e79687;tab=detail?project=oss-fuzz
spurious: ffmpeg libprotobuf-mutator libarchive irssi tinyusb naga htmlunit graphicsmagick fwupd fluxcd gstreamer igraph wolfssl wolfmqtt poppler ostree mapserver lxml libxslt libxml2
/gcbrun trial_build.py all --sanitizer coverage address memory undefined none
spurious: cloud-hypervisor fabric cryptofuzz minify miniz ygot suricata htmlunit bitcoin-core
real: wheel pycrypto proto-plus-python http-pattern-matcher libecc tinyusb
? wolfmqtt libecc wolfssl tensorflow cel-cpp
/gcbrun trial_build.py wolfmqtt libecc wolfssl tensorflow cel-cpp wheel pycrypto proto-plus-python http-pattern-matcher libecc tinyus bcloud-hypervisor fabric cryptofuzz minify miniz ygot suricata htmlunit bitcoin-core --sanitizer coverage address memory undefined none
If this breaks less than 10 projects in the end I think i will merge.
/gcbrun trial_build.py wolfmqtt libecc wolfssl tensorflow cel-cpp wheel pycrypto proto-plus-python http-pattern-matcher libecc tinyus bcloud-hypervisor fabric cryptofuzz minify miniz ygot suricata htmlunit bitcoin-core --sanitizer coverage address memory undefined none
/gcbrun trial_build.py wolfmqtt libecc wolfssl tensorflow cel-cpp wheel pycrypto proto-plus-python http-pattern-matcher libecc tinyus bcloud-hypervisor fabric cryptofuzz minify miniz ygot suricata htmlunit bitcoin-core --sanitizer coverage address memory undefined none introspector
Fuzz introspector depends on python3.9 right @DavidKorczynski ?
In general Python 3.9+ should be fine, but let me double check tomorrow if 3.10.8 has no issues and update if so.
One area where we depend on 3.9 is https://github.com/google/oss-fuzz/blob/daef594bce3f7b58ffd135772b50670f9aa0b9c2/infra/base-images/base-builder/compile_python_fuzzer#L33-L40 and this may take some bigger effort to resolve, because that logic relies on AST parsing which may (I think it did) change between 3.9 and 3.10.8. But I assume we can leave those lines as is?
/gcbrun trial_build.py wolfmqtt libecc wolfssl tensorflow cel-cpp wheel pycrypto proto-plus-python http-pattern-matcher libecc tinyus bcloud-hypervisor fabric cryptofuzz minify miniz ygot suricata htmlunit bitcoin-core --sanitizer coverage address memory undefined none introspector
@jonathanmetzman I took a slightly different approach at this in #12027. I've tested it locally with most of the projects that failed on this PR. Most of them seem to be passing, and I've flagged or drafted a patch for those that didn't.
Would you please take a look and let me know what you think whenever you get a chance?
Thanks!
After 5 years, 3.8 is EOL now: https://devguide.python.org/versions/#unsupported-versions
