Add posibility to assemble flat P2-repositories

[HannesWell]

Currently the tycho-p2-repository:assemble-repository mojo always produces P2-repositories with the default layout where binary, feature and plugin artifacts are located in the sub-directories named binary, features and plugins. But in some cases, e.g. if one wants to 'publish' a P2-repository as Github Release artifact a flat structure is necessary:

• https://fbricon.github.io/posts/use-github-releases-as-p2-repo/

In that case it would be handy if the tycho-p2-repository:assemble-repository mojo would have an option to build a flat repo instead where all files are in the same directory. From the default layout only all files from the binary, features and plugins directories have to be moved up to the root directory and the mappings element in the artifact.xml has to be adapted accordingly:

@fbricon maybe this will be interesting for you as well eventually.

[laeubi]

I'm using Github pages for release of P2 repos... also note that Tycho can consume zipped repos from maven coordinates now. In anycase the repo layout is not done by Tycho but P2 so maybe one wants to enhance it there first?

Beside that its of course not so hard to flatten the repo manually, but then one needs to make sure that plugins and features do not share the same id+version (what is possible).

[fbricon]

+100 for flat repos. Github pages are not meant to serve binaries, unlike Github releases. And you get download stats when using releases. But as @laeubi said, it's likely something that needs to be fixed 1st in p2 itself.

[laeubi]

BTW for a "flat" repo it might be a good ida to simply use the mapping rules:

• plugin_${id}_${version}.jar
• binary_${id}_${version}
• feature_${id}_${version}.jar

as an inspiration we already have https://tycho.eclipseprojects.io/doc/latest/tycho-p2-repository-plugin/remap-artifacts-to-m2-repo-mojo.html ... so maybe we can simply have a flatten-repo goal that rename the files and replace the rules... then we don't need any support from P2 directly.

[HannesWell]

I'm using Github pages for release of P2 repos...

I'm current also exploring pages for that, but it's either convoluted (as one cannot just add some artifacts AFAIK) or one has to commit the release artifacts to the repo on a separate branch. Both is not optimal.

also note that Tycho can consume zipped repos from maven coordinates now.

This was also discussed in Fred's post and the conclusion was that it's also not optimal as the zip is completely loaded each time the repo is contacted.

BTW for a "flat" repo it might be a good ida to simply use the mapping rules:

* plugin_${id}_${version}.jar

* binary_${id}_${version}

* feature_${id}_${version}.jar

That's a good idea in order to disambiguate file names, although I think the possibility of overlaps is low but not zero (and we should be sure).

as an inspiration we already have https://tycho.eclipseprojects.io/doc/latest/tycho-p2-repository-plugin/remap-artifacts-to-m2-repo-mojo.html ... so maybe we can simply have a flatten-repo goal that rename the files and replace the rules... then we don't need any support from P2 directly.

Initially I thought about just having another option for assemble-repository as one then doesn't have to worry about phases (package is already crowded), execution order (e.g. with the archiving), which then often requires to re-declare the configuration for repos that's otherwise generated by pomless.

In anycase the repo layout is not done by Tycho but P2 so maybe one wants to enhance it there first?

Later it can certainly also be interesting for the plain P2 MirrorApplication, but first I would like to start in Tycho to just move the files and adjust the mapping-rules element.

[laeubi]

I'm current also exploring pages for that, but it's either convoluted (as one cannot just add some artifacts AFAIK) or one has to commit the release artifacts to the repo on a separate branch. Both is not optimal.

What do you mean you not "adding some artifacts"?

That is the workflow I'm using:

https://github.com/cucumber/cucumber-eclipse/blob/main/.github/workflows/maven.yml

The pages then just look like this:

https://github.com/cucumber/cucumber-eclipse/tree/gh-pages?tab=readme-ov-file

Initially I thought about just having another option for assemble-repository as one then doesn't have to worry about phases

I don't think this is really suitable and complicates the mojo. At best we have one mojo for one task. So if I like to flatten the repo, adding another execution seems much more useful as it will work for any repository at any location.

which then often requires to re-declare the configuration for repos that's otherwise generated by pomless.

Pomless works best with the default, if you want to deviate from that using a pluginMnagement in the configuration-pom is both simple and usual.

[laeubi]

I think the possibility of overlaps is low but not zero (and we should be sure)

I fear it is not that unusual as one would think sometimes:

https://github.com/eclipse-pde/eclipse.pde/blob/de5de34f941425ef7364dce6ecbade695a185aa3/features/org.eclipse.pde-feature/feature.xml#L3 https://github.com/eclipse-pde/eclipse.pde/blob/de5de34f941425ef7364dce6ecbade695a185aa3/ui/org.eclipse.pde/META-INF/MANIFEST.MF#L4

[HannesWell]

I'm current also exploring pages for that, but it's either convoluted (as one cannot just add some artifacts AFAIK) or one has to commit the release artifacts to the repo on a separate branch. Both is not optimal.

What do you mean you not "adding some artifacts"?

There are upload-pages-artifact and deploy-pages actions to upload artifacts to page(s) directly, but each upload wipe-out the previous content entirely.

That is the workflow I'm using:

https://github.com/cucumber/cucumber-eclipse/blob/main/.github/workflows/maven.yml

The pages then just look like this:

https://github.com/cucumber/cucumber-eclipse/tree/gh-pages?tab=readme-ov-file

That's the other solution that requires you to add the artifacts to your git (to a separate branch, but still). The Kieler KLighD project does it like that as well: https://github.com/kieler/KLighD/

But let's not discuss the pro and cons of different deployment methods on GH and just assume there is a demand for a flat repo now :)

Initially I thought about just having another option for assemble-repository as one then doesn't have to worry about phases

I don't think this is really suitable and complicates the mojo. At best we have one mojo for one task. So if I like to flatten the repo, adding another execution seems much more useful as it will work for any repository at any location.

One additional parameter doesn't seem to add much complexity, but a separate mojo indeed allows to transform already existing repositories. So that's clearly an advantage.

which then often requires to re-declare the configuration for repos that's otherwise generated by pomless.

Pomless works best with the default, if you want to deviate from that using a pluginMnagement in the configuration-pom is both simple and usual.

In case I want to assemble a repo, have it flattened and then archived, I think it's very difficult or even impossible to have a new flatten-repository mojo executed between the assemble-repository archive-repository goal as both are bound to the package phase and part of the default lifecycle of repos. Maybe it works if one redefines the execution in a pluginMnagement section, but that's at least cumbersome. So that's the main disadvantage I see for a separate mojo.