portage
portage copied to clipboard
gentoo
sync: Add method to download zip archives
Add a simple method for synchronizing repository as a snapshot in a zip archive. The implementation does not require external utilities to download and unpack archive. This makes the method very cheap.
The main usecase being considered is obtaining snapshots of github repositories, but many other web interfaces for git also support receiving snapshots in zip format.
For example, to get a snapshot of the master branch:
https://github.com/gentoo/portage/archive/refs/heads/master.zip https://gitweb.gentoo.org/proj/portage.git/snapshot/portage-master.zip
or a link to a snapshot of the tag:
https://github.com/gentoo/portage/archive/refs/tags/portage-3.0.61.zip
Could you give a bit more context if possible (motivation and such)? I didn't realise it was interesting to try fetch snapshots of ebuild repositories.
The context of the problem is this: I was installing several new systems using gentoo stage3. I also have my specific overlay on github which is used for installation. But it turned out that gentoo stage3 does not contain either git or unzip. I have to constantly deliver git before adding overlay. The git package is not a very small program and it has dependencies on perl modules for example.
I decided to see what I could do about it and found out that my problem could be solved without external utilities at all. And this works for most git hosting.
I understand that the lack of git in stage3 is not a portage problem, but I thought it would be useful to have a sync methood that didn't require anything other than python.
I'd also say the gzip module might be better as AFAIK git forges always have .tar.gz too. The idea sounds fine to me though.
I'm not against tar.gz either. I was guided by the fact that github has a link to zip in the interface. For brunch, he can also make a tarball, but the link is only to the “download zip”. This is the only reason :)
@floppym I changed the base class to SyncBase, left only the sync method, and threw out the ".desc" file.
It came to my mind the ".desc" may be needed if we implement the ETag header check. But for now there is no such thing, it really isn’t needed yet.
I have implemented ETag processing. I may add this support as a separate commit to this PR, or it may be done sometime later.
I've only given it a cursory look since an earlier revision but I'm happy for it to go in once docs are updated.
Could you update the man pages, and also possibly add something to the wiki as well once this is merged?
I will try to add something to wiki when this PR will be merged.
Can someone merge it ?
@floppym It looks like this PR is just waiting for your review. Could you make it and put it apporve?
Wiki can be done later, but what about man pages? Just a small example in portage(5) should suffice (there's already others there).
Wiki can be done later, but what about man pages? Just a small example in
portage(5)should suffice (there's already others there).
@thesamesam I added the example to portge(5) as a separate commit.
Thank you both for the contribution and all for the reviews!
The patience is also appreciated. I'm sorry that I've been busy with a lot of other bits recently.
Sorry for the off-topic. I don't understand how I can change the page https://wiki.gentoo.org/wiki/Project:Portage/Sync . I have an account with a verified email, but I can only open a discussion.
Add the diff to discussion, a moderator or project dev will need to apply it.
