aiohttp icon indicating copy to clipboard operation
aiohttp copied to clipboard

pip-audit flagging aiohttp: https://github.com/advisories/GHSA-rwqr-c348-m5wr

Open jrobbins-LiveData opened this issue 2 years ago • 26 comments

Describe the bug

pip-audit is flagging aiohttp as having a Moderate vulnerability, apparently due to https://github.com/aio-libs/aiohttp/issues/6772.

Found 1 known vulnerability in 1 package Name Version ID Fix Versions


aiohttp 3.8.1 GHSA-rwqr-c348-m5wr

To Reproduce

  1. Install latest pip-audit
  2. run it (e.g. pip-audit) in an environment that has the latest aiohttp installed

Expected behavior

The pip-audit results should not flag aiohttp as having a security vulnerability.

Logs/tracebacks

Found 1 known vulnerability in 1 package
Name    Version ID                  Fix Versions
------- ------- ------------------- ------------
aiohttp 3.8.1   GHSA-rwqr-c348-m5wr

Python Version

$ python --version

Python 3.9.12

aiohttp Version

$ python -m pip show aiohttp

Name: aiohttp
Version: 3.8.1
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author:
Author-email:
License: Apache 2
Location: c:\users\jeff1\documents\projects\git\config-extension\.venv\lib\site-packages
Requires: aiosignal, async-timeout, attrs, charset-normalizer, frozenlist, multidict, yarl
Required-by: aioresponses, config-extension, extension-base

multidict Version

$ python -m pip show multidict

Name: multidict
Version: 6.0.2
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache 2
Location: c:\users\jeff1\documents\projects\git\config-extension\.venv\lib\site-packages
Requires:
Required-by: aiohttp, yarl

yarl Version

$ python -m pip show yarl

Name: yarl
Version: 1.7.2
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache 2
Location: c:\users\jeff1\documents\projects\git\config-extension\.venv\lib\site-packages
Requires: idna, multidict
Required-by: aiohttp

OS

Edition Windows 10 Pro Version 21H2 Installed on ‎5/‎1/‎2021 OS build 19044.1766 Experience Windows Feature Experience Pack 120.2212.4180.0

Related component

Server, Client

Additional context

No response

Code of Conduct

  • [X] I agree to follow the aio-libs Code of Conduct

jrobbins-LiveData avatar Jun 25 '22 13:06 jrobbins-LiveData

I thought that issue is just that a ValueError is being thrown. I feel like an app is poorly designed if throwing a ValueError can cause a DoS. Maybe I'm misunderstanding something, but I'm not seeing why this has been assigned a moderate vulnerability rating.

@webknjaz Has this already been highlighted to you via email? I'm not listed on the security policy, so maybe there's more information that I've not seen.

Dreamsorcerer avatar Jun 25 '22 18:06 Dreamsorcerer

I haven't seen anything related in my inbox. So far, I don't see any evidence of a security issue in the submitted issues 🤷‍♂️.

webknjaz avatar Jun 25 '22 22:06 webknjaz

Perhaps one of the project's devs could email [email protected] and ask what the security issue is? The status given at https://nvd.nist.gov/vuln/detail/CVE-2022-33124 is

This vulnerability is currently awaiting analysis.

which makes me wonder what their process is, as it seems to cause unnecessary concern to list something as a "vulnerability" without giving any more context than

aiohttp v3.8.1 was discovered to contain an invalid IPv6 URL which can lead to a Denial of Service (DoS).

jrobbins-LiveData avatar Jun 25 '22 22:06 jrobbins-LiveData

@jrobbins-LiveData I have a strong feeling that some random GitHub user created that advisory on GitHub and since GitHub can issue CVE numbers, it did so. But without any detail, it just doesn't look credible, it seems to just be a baseless speculation. For now, I suggest to treat is as an invalid report, until proven otherwise.

aiohttp v3.8.1 was discovered to contain an invalid IPv6 URL which can lead to a Denial of Service (DoS).

This description looks rather bizarre to me. A person experienced with describing security vulnerabilities wouldn't make such silly mistakes. What does "aiohttp contains invalid IPv6 URL" even means? It doesn't "contain" URLs — the end-user apps/configs may. It's a framework, after all.

webknjaz avatar Jun 26 '22 19:06 webknjaz

FYI https://security.snyk.io/vuln/SNYK-PYTHON-AIOHTTP-2934978

giorgiop avatar Jun 26 '22 19:06 giorgiop

@webknjaz according to the NIST website, the CVE "Source" is "MITRE", which is not a random user.

The website has a feedback link, so I submitted feedback (choosing "Request an update to an existing CVE Entry") requesting that that odd phrase "aiohttp contains invalid IPv6 URL" be clarified and offered my opinion that there was no issue here.

I received this auto-response:

Thank you for the following submission:

CVE IDs to be updated:  CVE-2022-33124


It will be reviewed by a CVE Assignment Team member. Changes, additions, or updates to your request can be sent to the CVE Team by replying directly to this email.
Please do not change the subject line, which allows us to effectively track your request.

CVE Assignment Team

M/S M300, 202 Burlington Road, Bedford, MA 01730 USA

[A PGP key is available for encrypted communications at

http://cve.mitre.org/cve/request_id.html]

{CMI: MCID13436943}

While I agree 100% that this issue looks to be a misunderstanding, and this a pseudo-issue, and wonder what the process control checks are on this CVE process, I don't think we can simply ignore it. Rather, I think we need to get it corrected or expunged.

I strongly urge you (I'm presuming you are one of the qualified devs of this repo?) to also file a report at https://cveform.mitre.org/ or take any other formal measures to get this report removed.

jrobbins-LiveData avatar Jun 26 '22 19:06 jrobbins-LiveData

FYI https://security.snyk.io/vuln/SNYK-PYTHON-AIOHTTP-2934978

Interesting, although it doesn't seem to provide any proof or explanation either.

While I agree 100% that this issue looks to be a misunderstanding, and this a pseudo-issue, and wonder what the process control checks are on this CVE process, I don't think we can simply ignore it. Rather, I think we need to get it corrected or expunged.

Well, I've started here: https://github.com/github/advisory-database/pull/444 / https://github.com/github/advisory-database/pull/445.

@webknjaz according to the NIST website, the CVE "Source" is "MITRE", which is not a random user.

Yeah, it's weird that it's coming from MITRE. It's true that the last time I filed a CVE through GitHub it showed up as coming from GitHub, Inc.

I strongly urge you (I'm presuming you are one of the qualified devs of this repo?) to also file a report at https://cveform.mitre.org/ or take any other formal measures to get this report removed.

Yes, I am one of the core maintainers. But I've been having a lot on my mind recently with not much energy for FOSS so I haven't been as involved these these recent months. I'll see what I can do.

webknjaz avatar Jun 26 '22 19:06 webknjaz

@webknjaz I truly apologize for not knowing your circumstances. Please let me know if I can do any of the leg work in tracking down the process at Mitre that led to this.

jrobbins-LiveData avatar Jun 26 '22 19:06 jrobbins-LiveData

On re-reading the issue https://github.com/aio-libs/aiohttp/issues/6772#issue-1253751995, I see that the issue has this phrase in it

Denial of service

Perhaps MITRE is using some bot and surfaced this issue? It might make sense to address this issue and close it?

jrobbins-LiveData avatar Jun 26 '22 20:06 jrobbins-LiveData

@webknjaz I truly apologize for not knowing your circumstances. Please let me know if I can do any of the leg work in tracking down the process at Mitre that led to this.

No worries, thanks for understanding! And yes, it'd be helpful if somebody more familiar with how MITRE operates could shed some light on why this happened.

On re-reading the issue #6772 (comment), I see that the issue has this phrase in it

Denial of service

Perhaps MITRE is using some bot and surfaced this issue? It might make sense to address this issue and close it?

That's an interesting idea. Although it never occurred to me that spam CVEs are a thing. If the bot theory is correct, this would probably mean that the process is rather broken — it seems like GitHub is marking the CVEs as manually reviewed: https://github.com/github/advisory-database/commit/f550c16e12586ca8821e94dbbc7c116bec4c6788#diff-bb93cb225bc1240a376f8a9f507326679be13265e96894dd78488990d8ba2826. This has an implication of "a human looked at the thing and decided it's valid". But if they're blindly exporting the data and auto-add a "seal-of-approval", well that'd indicate more problems in the pipeline than I originally anticipated...

And yes, it always does make sense to address the bugs. But that issue does not really describe a bug — it does not demonstrate any reproducer code, nor does it clearly state why the reporter thinks it's a problem. So it's rather hard to resolve something that would only be based on a series of guesses nobody is able to verify.

webknjaz avatar Jun 27 '22 09:06 webknjaz

I just noticed that there's an update there:

CVE Modified by MITRE 6/26/2022 4:15:08 PM

Now it says:

** DISPUTED ** AIOHTTP 3.8.1 can report a "ValueError: Invalid IPv6 URL" outcome, which can lead to a Denial of Service (DoS). NOTE: multiple third parties dispute this issue because there is no example of a context in which denial of service would occur, and many common contexts have exception handing in the calling application.

(emphasis mine)

webknjaz avatar Jun 27 '22 11:06 webknjaz

Section C5 of this document explains the ** DISPUTED ** status terminology, and section 9 describes an "Appeals" process. I'm glad that we can see that ** DISPUTED ** status, as it shows that there is some human oversight in the process. I'll try to find out more about how we can request that this spurious report be REJECTED.

jrobbins-LiveData avatar Jun 27 '22 12:06 jrobbins-LiveData

So it seems that per the following point of section 7.1, the CVE should've never been assigned:

7.1.1 If a product owner considers an issue to be a vulnerability in its product, then the issue MUST be considered a vulnerability, regardless of whether other parties (e.g., other vendors whose products share the affected code) agree.

(since nobody actually attempted to have a dialogue with us, to the best of my knowledge)

I wonder if I email them too, will they update that "multiple third parties" to "the core developers"?

webknjaz avatar Jun 27 '22 12:06 webknjaz

Section 9 outlines an appeals process:

The party seeking to appeal a decision made by a Root, or resolve a disagreement between Roots, contacts their hierarchy's Top-Level Root. For example, the MITRE Top-Level Root would be contacted at https://cveform.mitre.org/, and the following steps would be followed: Enter your email address. Select “Other” in the Select a request type field. Select “Issue” in the Type of comment field. Enter “Arbitration Request”, in the Please provide your question, issue, comment, etc. field.

This appears to be a formal way you can weigh in. If you have time, I recommend you do so as soon as possible.

jrobbins-LiveData avatar Jun 27 '22 12:06 jrobbins-LiveData

Ah, I've just submitted that form but with the "Update request" selected. There was a "Reject CVE" which I selected in a drop-down that appeared later.

webknjaz avatar Jun 27 '22 13:06 webknjaz

I suggest making a second submittal following their procedure. I see in "7.1.2 If the CNA determines that an issue violates the security policy of a product, then the issue SHOULD be considered a vulnerability." so it is possible that the CNA (MITRE in this case) made the determination. I am sending them a question right now as to whether or not they made the determination.

But I think the shortest path to resolution is for you to make the "Arbitration Request" explicitly, so that we can follow their process and get this issue resolved.

jrobbins-LiveData avatar Jun 27 '22 13:06 jrobbins-LiveData

Okay, I've just submitted that too.

webknjaz avatar Jun 27 '22 13:06 webknjaz

Hey all 👋🏽 I'm Idan from Snyk security team.

I would like to mention that we are fully aware of this discussion and familiar with the current situation.

We removed this vulnerability from our database and revoked the specific advisory. (As for now, the advisory is still publicly available but will be removed in a few hours as part of a general publication process.)

Idan-D avatar Jun 27 '22 14:06 Idan-D

Thanks for the update @Idan-D!

webknjaz avatar Jun 27 '22 14:06 webknjaz

pip-audit maintainer here: once the advisory is purged from GHSA and other DBs, pip-audit should stop displaying it. I'm coordinating with the PYSEC database maintainers now to ensure that we prune this, if appropriate.

woodruffw avatar Jun 27 '22 14:06 woodruffw

@woodruffw thanks for the update!

webknjaz avatar Jun 27 '22 14:06 webknjaz

No problem, many thanks to @alex for bringing it to my attention!

I've posted a temporary "workaround" for pip-audit users here: https://github.com/aio-libs/aiohttp/issues/6772#issuecomment-1167464922

woodruffw avatar Jun 27 '22 15:06 woodruffw

UPD: At the moment, this advisory is marked as withdrawn in GHSA and Snyk lists. And it still shows up as disputed on the NIST website.

webknjaz avatar Jun 29 '22 13:06 webknjaz

pip-audit is also no longer reporting this:

pip-audit --no-deps -r <(echo 'aiohttp==3.8.1')
No known vulnerabilities found

woodruffw avatar Jun 29 '22 14:06 woodruffw

UPD: NIST still haven't withdrawn the CVE, MITRE haven't replied either. UPD (Nov 22, 2022): Nothing's changed so far.

webknjaz avatar Jul 09 '22 18:07 webknjaz

NIST still haven't withdrawn the CVE, MITRE haven't replied either.

As per CVE's FAQ, It seems that NIST cannot withdraw the CVE but only MITRE can do:

The CNA that published the CVE Record must be contacted to request an update. Refer to the List of Partners page for CNA contact information.

Also, MITRE does not monitor their [email protected] email address. But after receiving an auto-sent confirmation email with a reference number for requests that were submitted on https://cveform.mitre.org (the only way to get initial contact with MITRE), any reply with the reference number is monitored:

Use the CVE Program Request forms for ALL inquiries to the CVE Program. If you are unsure which option to choose in the dropdown menu, select “Other.” Once submitted, requesters receive an email confirmation message sent from [email protected] with a reference number, which the CVE Program uses for request processing and tracking purposes. If encrypted communications are required, click here.

CVE Program email accounts are NOT monitored; only REPLY email messages with a reference number are processed by the [email protected] email account.

Note: Contact methods for individual CVE Numbering Authorities (CNAs) and Roots are provided on the List of Partners page.

My advice is that the only thing we can do now is to nudge MITRE by replying to their confirmation email. Or, maybe, contact GitHub to ask their staff to make a request to see if their requests are of higher priority?

Rongronggg9 avatar Aug 22 '22 01:08 Rongronggg9

Even Dependabot is still reporting this in our own repos: image

Doesn't seem like there's anything more we can do now, so let's just close this issue.

Dreamsorcerer avatar Nov 22 '22 11:11 Dreamsorcerer

Even Dependabot is still reporting this in our own repos:

It says

Opened 5 months ago

so it was probably created right before I asked GitHub to withdraw it in and hasn't been updated after the status change. I suppose Dependabot may be not updating the alerts post creation. @shelbyc do you know if that's by design?

webknjaz avatar Nov 22 '22 15:11 webknjaz

@webknjaz Thank you for reporting a withdrawn advisory with open dependabot alerts! I have alerted the appropriate team and they are planning to fix the bug.

shelbyc avatar Nov 28 '22 20:11 shelbyc