Update requests to fix CVEs (security)

[jk464]

Fixes up a host of CVEs in the st2 package:

Note: The XRAY references are vulnerabilities listed by JFrog Xray, that don't seem to have a corresponding CVE. JFrog doesn't seem to publish these references publicly - but I've linked to the issue disclosing the vulnerability thats referenced by the XRAY entry.

Bump cryptography to 41.0.4, pyopenssl to 23.2.0

Fixes:

• CVE-2023-4807
• CVE-2023-2650
• CVE-2023-3446

pyopenssl 23.2.0 required for cryptography to 41.0.x support

Bump virtualenv to 20.16.7

Fixes:

• CVE-2019-20916

Bump importlib-metadata to 4.10.1

Fixes:

• XRAY-195083

Bump requests to 2.31.0

Fixes:

• CVE-2023-32681

Bump gitpython to 3.1.37

Fixes:

• CVE-2023-40267
• CVE-2023-41040
• CVE-2023-40590
• CVE-2022-24439
• XRAY-198950

Supercedes/Implements

• https://github.com/StackStorm/st2/pull/6059
• https://github.com/StackStorm/st2/pull/6058
• https://github.com/StackStorm/st2/pull/6057
• https://github.com/StackStorm/st2/pull/6056
• https://github.com/StackStorm/st2/pull/6054
• https://github.com/StackStorm/st2/pull/6053

[jk464]

@armab As you can probably see from my commits I've hit a bit of a depedency hell trying to get requirement ranges that:

• Allow the last supported py3.6
• A version with relevant CVEs fixed in py3.8

I can see in https://github.com/StackStorm/st2/pull/6063 you've hopefully got gitpython handled.

I'll probably do the same as you here and split this into bit size PRs to make it more manageable.

I did look at fixing fixate-requriements.py to support reading python_version which I actually got working (See https://github.com/StackStorm/st2/pull/6062/commits/61b47db603c7d21a06142228f031713ba77d7a35) (which is probably worth having, even after dropping py3.6 in 3.9.0) - issue I hit was the stackstorm/packagingbuild:bionic was missing the package that I wanted to using (packaging) so that image would also need bumped.

If you think its worth our time adding that support, I'll take a look at updating the image as well :)

[arm4b]

@jk464 Your enhancement to fixate-requriements.py looks really clean. But overall I felt like env markers are buggy in many places, including older pip version we're locked to (because of py3.6) and even pants that doesn't support them in requirements-pants.txt so touching them might be like opening a can of worms. I hope the builds would be migrated to pants from all the old machinery and so fixate-requriements.py may be obsolete by then.

@cognifloyd do you think it's doable to migrate to the pants builds in the upcoming v3.9.0?

[arm4b]

So at least importlib-metadata changes are extracted into a dedicated PR which could be merged ASAP: https://github.com/StackStorm/st2/pull/6072

[jk464]

@armab if the build is failing for requests and the update to importlib-metadata was merged in #6072 is there anything left to do in this PR?

The only CVE I see listed against requests is CVE-2023-32681 - which is only medium Sev - so I think we could just drop fixing that to 3.9.0 at which point we can(?) drop python 3.6 and bump this w/o issue. (And I assume for 3.9.0 we'd probably rather being bumping dependencies to the highest support by 3.8.)

Let me know what you think and I can close this PR if there's nothing further to do

[arm4b]

@jk464 Yeah, let's reassign this PR to the v3.9.0 roadmap It'll be a reminder that requests will need an update after dropping the py3.6.