Upload Pypi artifacts from Github Actions as prerelease

[daddycocoaman]

Is your feature request related to a problem? Please describe. Hi, I am using volatility 3 as a dependency for a tool I've written and I would like to upload it to PyPI without additional actions required from the user to getting the develop version of vol3.

Somewhere between 2.0.1 and the current 2.2.0, something was fixed that addressed Page Fault at entry 0x0 errors I was getting. In development, I was using the current 2.2.0 as found in develop. Rolling back to 2.0.1 breaks my tool.

Describe the solution you'd like It looks like there was discussion about uploading releases to PyPI here https://github.com/volatilityfoundation/volatility3/pull/438 but it looks like the Github Action solution worked for what was needed. My request is that builds from develop that are made with the Github Action be uploaded to PyPI as Pre-release packages. The assumption here is that if you've moved up a minor version on Github, it makes sense to release a version of it on PyPI. I also understand it's not the "stable" version, but the normal user will still get the stable version on PyPI (2.0.1 at this time) while others like myself can opt into the prerelease version for users.

Describe alternatives you've considered I've added additional instructions to my tool so users can get the vol3 dependency themselves but it's significantly easier just to have to available on PyPI. It currently looks like this:

pipx install dumpscan
pipx inject dumpscan git+https://github.com/volatilityfoundation/volatility3#39e812a

[digitalisx]

I think PyPI and Github Release should provide the same version. Can many of the communities agree with the PreRelease on PyPI and, if possible, consider PreRelease in Github Release?

[ikelos]

Hi there, I'm sorry it took us so long to cover this issue, but we've now discussed it amongst the group. The feeling is that we're not keen on machinery automatically rolling releases for us from the repo and pushing it to the official pypi, but we also realize we're not that fast as rolling releases (although we have branched for one recently which is currently scheduled for a December release). So, we've decided to try and increase the frequency with which we roll releases, and we're shifting from a twice yearly schedule to aim for a release every 4 months. Hopefully this will reduce the window that people might be waiting on a fix, whilst still ensuring we don't have problems by automatically publishing a release that's broken for some reason to pypi...

As such I'm going to mark this as closed, but feel free to reopen it if you think the issue needs more consideration.

[daddycocoaman]

Hi,

Thanks for responding. There's a difference between an official release and a pre-release on PyPI. The semiannual or every 4 months release could be an official one, while the Github Action release could be a PyPI prerelease, so even if something does breaks, regular users won't get it unless they specifically ask for it.

My personal opinion is that a bump in the minor version would warrant a new release regardless of elasped time (i.e., 2.0.1 is still the current released version but you're actually up to 2.4.0). As a standalone tool, that's might fine because you've the recommended way for installation but as a dependency, it just feels weird.

I guess this issue has also been somewhat mitigated by some dependency managers, where you can specify the branch or commit to use. As a dependency, the current approach is a little more to handle because I'd have to manually (or automated) check when a new tag is set and manually update dependencies each time to ensure my tool is latest and greatest. Could definitely be automated, just a lot more to maintain.

Thanks anyway, though.

[ikelos]

The reason for not bumping every minor release is somewhat due to how we do versioning, Chrome and Firefox go up a major version each release, but go up by thousands on their point releases, for example and they don't push thousands of versions (granted, they're not often used as dependencies either). We're trying hard to follow the semantic versioning guidelines but there's not requirement that every point release gets published (and given how frequently point releases may occur when sticking to the rules, it would be a little infeasible).

So the options we have would be:

• don't bump the version number in development (breaks semantic versioning for development code)
• pump out hundreds of releases for every API visible change to the code (time consuming)
• have official releases on a schedule and hope that the gap between a feature and a release isn't too large

We've opted for the third one, and hopefully, due to the version numbers being increased every time there's an API visible change, your code can determine whether the version installed is the right one or not. I think the burden of supporting the prereleases and the bug reports we'd get based on normal users wanting the "latest and greatest" would be too much for us to handle. Automation or not, we have to respond to bugs people raise against the code even if we'd class it as a prerelease.

I hope that all makes sense, and I'm glad at least you can use specific commit IDs to get the version you need immediately before the next release is rolled. Hopefully we'll get the window down enough that it's a good compromise for both sets of needs...