Publish multiple snapshots to the same prefix/distribution ; sharing the same `/pool`

[kumy]

trafficstars

I know we're warned about such a config, but it could have advantages.

"It is not allowed to publish two repositories or snapshots to the same prefix and distribution."

(Here we used to run reprepro for our internal mirrors. And we're planning moving to aptly)

We were able to provide daily snapshots of mirrors, each of them sharing the same /pool.

$ reprepro gensnapshot trusty  20150125
$ reprepro gensnapshot trusty  20150126
$ reprepro gensnapshot trusty  20150127

$tree -d dists
dists/
`-- trusty
    |-- main
    |   `-- binary-amd64
    `-- snapshots
        |-- 20150125
        |   `-- main
        |       `-- binary-amd64
        |-- 20150126
        |   `-- main
        |       `-- binary-amd64
        `-- 20150127
            `-- main
                `-- binary-amd64

I've found that I can publish daily snapshots in different subdirs, ie:

aptly publish snapshot  -component="main" ubuntu-security-trusty_20150207 ubuntu-security/snapshots/20150207
aptly publish snapshot  -component="main" ubuntu-security-trusty_20150208 ubuntu-security/snapshots/20150208

But this has (for us) the real disadvantage of multiplying /pool directories.

What's not "cool" with that is, that we use caching proxies. Many servers depends on different snapshots. Caching is not efficient, as the storing hash key is the .deb full path.

If all servers download, let's say the same openssl_[...].deb from differents snapshots, then we'll not be able to serve cached content from the 1st request, we'll cache multiple times the same .deb (as of md5 speaking).

I tried to "play" with the -distribution argument, but no success

$ aptly publish snapshot -distribution="trusty/snapshots/20150207" -component="main" ubuntu-security-trusty_20150208 ubuntu-security                      
ERROR: unable to publish: invalid distribution trusty/snapshots/20150207, '/' is not allowed

Something like that seem to be related to issue #115

Is there another method for doing such thing ? Do you think it could be implemented ? How could I help you ?

---

Reprepro doc on gensnapshot:

 gensnapshot codename directoryname

 Generate a snapshot of the distribution specified by codename in the directory dists/codename/snapshots/directoryname/ and reference all needed files in the pool as needed by that.

aptly 0.9~dev+180+g7925af9

[kumy]

As an alternative we could also publish daily snapshots as:

$ aptly publish snapshot -distribution="trusty-snapshots-20150207" -component="main" ubuntu-security-trusty_20150207 ubuntu-security
$ aptly publish snapshot -distribution="trusty-snapshots-20150208" -component="main" ubuntu-security-trusty_20150208 ubuntu-security

tree -d dists/
dists/
|-- trusty
|   `-- main
|       |-- binary-all
|       |-- binary-amd64
|       `-- binary-i386
|-- trusty-snapshots-20150207
|   `-- main
|       |-- binary-all
|       |-- binary-amd64
|       `-- binary-i386
`-- trusty-snapshots-20150208
    `-- main
        |-- binary-all
        |-- binary-amd64
        `-- binary-i386

[smira]

@kumy in short aptly doesn't support distribution names with / in it. All published repositories under the same prefix share pool directory.

The difficulty with / in distribution names is possible complexity with directory handling for such case. Most probably it is possible to implement that, but I need to investigate this case

[kumy]

The alternative method I proposed doesn't work with debootstrap

$ fakeroot debootstrap trusty-snapshot-20150209 mytest http://localhost/distrib/ubuntu/
E: No such script: /usr/share/debootstrap/scripts/trusty-snapshot-20150209

[evrardjp]

This is also what we are looking for.

Is there a way to do this?

I investigated the use of components for this, but then if a snapshot is bad, deleting experience is horrible:

• deleting only a component of a publication is not possible, swapping is painful because we have to move with an intermediate snapshot.
• re-creating a new component on top of a current publication is painful (you cannot have aptly publish ... -component=new-snapshot-version ... because it will consider the distro is already published.

It seems impossible to have a versioned repo that way.

[evrardjp]

No comment on closing?

[lbolla]

@evrardjp mmm I suspect this issue was closed by mistake, as it has a relatively recent comment (yours).

[neolynx]

I think this is possible in the latest version with the -multi-dist option

[neolynx]

Aptly 1,6,0 has been released: https://github.com/aptly-dev/aptly/discussions/1414

I believe the issue is fixed, please reopen if it still persists...

[kumy]

I'm not using aptly anymore, so can't confirm. Thanks for adding it anyway 😉