bandersnatch
bandersnatch copied to clipboard
pypa
Add/test `delete` + `verify` support for non 'filesystem' mirrors
Delete and verify are not known to support anything other than filesystem well today. We should add swift and any other filesystem support and have a little more testing somehow.
not sure if it's relevant, I'm using s3 storage to mirror packages, but the packages deleted in pypi.org was not deleted in my mirror sites. what can I do to troubleshoot this?
Sure is relevant. A mirror run has no idea what to delete. In order to work that out bandersnatch has to do a full sync and filesystem crawl to then compare what was synced and what was not to workout what to delete. Not an efficient or cheap process at all.
Without PyPI API enhancements, we don't have a good way here to workout what to delete periodically.
Is there any existing command to achieve this? Or I have to implement it on my own?
This does not exist. Ideally to do this, we need to refactor verify (and delete) to use the storage abstractions to write to their respective back ends and mold the current state maintain by each sub command.
Could this be considered done since https://github.com/pypa/bandersnatch/pull/1142 is merged?
Sure. I just haven't used it with non file system mirrors so didn't know if we'd covered it all. Happy to call it done due to your PR :)