Security Policy needs an overhaul

[pcaversaccio]

The current Security Policy needs a proper overhaul imho since the content is outdated (e.g. Vyper is not currently in limited beta; Yearn, Curve Finance, or Lido have conducted extensive audits on Vyper contracts). Also linking here the write-up by @fubuloubu about the "Vyper Security Alliance". There is no rush to update the Security Policy, but I personally would aim for a properly overhauled version for the major 0.4.0 release.

Information that needs to be included in a Security Policy is imho:

• Summary of all known security vulnerabilities (link to a specific section in the docs; not only the security advisories);
• List of security vulnerabilities affecting a specific version of the compiler (can be e.g. a JSON file);
• What's the policy around patching versions? E.g. let's assume there is a bug found in Vyper 0.3.9 which is also present in 0.2.16, does Vyper release a patch not only for the 0.3.x series, but also 0.2.x series?;
• Is there a bounty program we can somehow include and know how to fund?;
• ...

I opened this issue to gather all the feedback in one place. Let me know your thoughts.

[MatthiasEgli]

For the bug bounty program, I suggest to for now add that the Vyper Security Alliance would handle coordination with affected projects and any bug bounty rewards due to that would be given to the whitehat, basically adding this as an indirect bug bounty program.

[fubuloubu]

What's the policy around patching versions? E.g. let's assume there is a bug found in Vyper 0.3.9 which is also present in 0.2.16, does Vyper release a patch not only for the 0.3.x series, but also 0.2.x series?

Unfortunately, we cannot patch old versions, namely because once contracts are compiled with a version of vyper, there's no way we do anything about it when a bug is found. Best we can do is advisory at that point, so people are aware of any potential issues with the version they have used to deploy with.

We have occasionally yanked releases if they are less than a few days old, but I think the practice moving forwards (@charles-cooper correct me if I'm wrong) is to do release candidates before each release to make sure the types of integration bugs we have typically found in the past which led to a yank get discovered before a formal tag is made.

The versioning guideline in the docs should also be updated to mention this

[charles-cooper]

We have occasionally yanked releases if they are less than a few days old, but I think the practice moving forwards (@charles-cooper correct me if I'm wrong) is to do release candidates before each release to make sure the types of integration bugs we have typically found in the past which led to a yank get discovered before a formal tag is made.

yep -- we did this for v0.3.10 and it worked quite well for finding these kinds of integration issues pre-release

[pcaversaccio]

Unfortunately, we cannot patch old versions, namely because once contracts are compiled with a version of vyper, there's no way we do anything about it when a bug is found. Best we can do is advisory at that point, so people are aware of any potential issues with the version they have used to deploy with.

I think I need to clarify here what I meant by patch: i.e. a new Vyper patch version in a 0.X.Y series. Let's say we find a bug in the 0.3.10 version that affects a built-in function. The bug is also persistent in the old 0.2.X versions. Now the question is do we also release a new 0.2.17 version or only a 0.3.11 version that fixes the bug? I guess we will not do the former, but we have to document this properly still.

[fubuloubu]

Unfortunately, we cannot patch old versions, namely because once contracts are compiled with a version of vyper, there's no way we do anything about it when a bug is found. Best we can do is advisory at that point, so people are aware of any potential issues with the version they have used to deploy with.

Now the question is do we also release a new 0.2.17 version or only a 0.3.11 version that fixes the bug? I guess we will not do the former, but we have to document this properly still.

Correct, no patching older compiler versions

[pcaversaccio]

Reopening since it got closed accidentally by the merge of https://github.com/vyperlang/vyper/pull/3981.