Multiple requests for GSoC 2024 Plugin Modernizer Tool

[jonesbusy]

Service(s)

Other

Summary

Hi,

With GSoC 2024 coming to an end we would like to industrialize the Plugin Modernizer Tool we developed during the GsoC 2024 period

Right now we have a repository https://github.com/jenkinsci/plugin-modernizer-tool and this is fine for the code.

This tool has mainly 2 axis

1. Getting metadata about plugins (we are using an OpenRewrite scanning recipe for this). For example we can extract information about transitive dependencies or just to know if a specific file is present
2. Running recipe and refactoring and open a pull request with the transformation

For the 1) we would like to store this information (it's basically JSON document generated by the tool) on a Git repository. Historically some maintainer were using their personal repos to store this information (example https://github.com/gounthar/jdk8-removal) Now we would like to have a central place to store those JSON document.

What is the best organisation for this ? jenkinsci or jenkins-infra ?

In a more long term future (perhaps a future GSoC project) those metadata could be consumed by the plugin health score system to create more probe that cannot be created without help of OpenRewrite (like getting transitive dependencies and warn if a plugin doesn't use an API plugin instead).

In a short term only the maintainers of the tool would push those metadata to this repository until we have some kind of workflow/job to do it for us

The second point is about the distribution of the CLI. We would expect some frequent release and we need to think about packaging and would like perhaps to start with HomeBrew distribution to cover Linux and MacOSX user

I noticed you guys are using already a Tap on https://github.com/jenkins-infra/homebrew-tap/. Is it something we could also use to distribute our CLI ? Or do you see it more on the jenkinsci organisation ?

Thanks for taking the time to answer with your thought.

Regards,

Reproduction steps

No response

[jonesbusy]

FYI @gounthar @sridamul

[timja]

Jenkins Infra GitHub org would make most sense to me as its information and tooling for the Jenkins project and not for end users of Jenkins.

and it related to plugin health score and plugin site as well with the potential integration points.

[dduportal]

+1 with @timja

• About the CLI: setting up the jenkins-infra homebrew TAP looks good! We use GitHub action for this. Until now, we only had Golang CLI which uses the GoRelease tool (example here: https://github.com/jenkins-infra/jenkins-version/blob/7135e0c185f61fa9c88cd782f70ac0144cf0b611/.goreleaser.yml#L63-L76) but we can totally use Jenkins (infra.ci.jenkins.io) if needed. Never published a CLI tools to homebrew though: will be a first \o/

  • It means we'll have to transfer the repository https://github.com/jenkinsci/plugin-modernizer-tool to the https://github.com/jenkins-infra organization (ping @timja does it look good to you?) as a start

• About the JSON data to store, you have the following choices:

  • No problem to use a (distinct) repository in jenkins-infra as well IF the data history is needed as a convenience (and is hard to rebuild).
    • Then, either @gounthar is Ok to transfer https://github.com/gounthar/jdk8-removal,
    • Or we can create a new repo in jenkins-infra: in this case can you share with us the expected repo name and the maintainer to add to it?
  • Otherwise, if you only need a "publish the latest version", we can use to https://reports.jenkins.io/ like Plugin Health or other repositories are doing. The JSON would be reachable through a simple webserver (any GET HTTP request would suffice).

[jonesbusy]

Hi,

Looks good for me. We can transfer the repository to jenkins-infra as long we can keep our maintainers permission.

I also never published anything to homebrew, right now the GH action just use the cd.yaml workflow.

https://jreleaser.org/ looks a good alternative for Java project. They are some module to publish to HomeBrew.

The history of metadata is not so important, the last option would also be possible, how could we publish such JSON files to this WebServer. Do you have any example ?

I've craeted 2 issues related to packaging

https://github.com/jenkinsci/plugin-modernizer-tool/issues/239 https://github.com/jenkinsci/plugin-modernizer-tool/issues/240

(Perhaps for Hacktoberfest)

[gounthar]

For Homebrew, @jmMeessen has already published for Jenkins in https://github.com/jenkins-infra/jenkins-contribution-aggregator thanks to goreleaser, but I don't know what would be the equivalent (if any) for Java. 🤔

For the docker image, we could publish on GitHub as a package, as we did with the quickstart tutorials

[timja]

It means we'll have to transfer the repository jenkinsci/plugin-modernizer-tool to the @jenkins-infra organization (ping @timja does it look good to you?) as a start

Looks fine to me.

[jonesbusy]

but I don't know what would be the equivalent (if any) for Java. 🤔

Yes there is (https://jreleaser.org/) but I never played with it

[dduportal]

Hello folks, we have delayed any work on this task to mid-October as the infra team will be in limited availability

[jonesbusy]

An other point (that can be discussed on an other issue is about a GitHub for the CI (GH action or Jenkins infra I don't have strong opinion)

We are adding the support on https://github.com/jenkinsci/plugin-modernizer-tool/pull/295 in addition of GH_TOKEN (that is used when running the CLI from a maintainer machine)

We still need to define granular permissions, but the CLI will soon be able to use GitHub app installation in order to open PR

[dduportal]

Hello folks, back at this topic.

Whenever you are ready, you can transfer the repository(ies) to the GH organization jenkins-infra.

Please let us by commenting here once you did it, and tell us the expected maintainers so I can add them back if the transfer changes user permissions.

Then, we'll start working with issues in the repository(ies) for the tasks

• CLI publication (homebrew tap + Docker images)
• CI system (ci.jio?)
• CD system (GH app? infra.ci.jio ?)

[jonesbusy]

Hi,

Thanks for the update.

I personnally cannot perform the transfer because it must be initiated by an org owner of jenkinsci

Perhaps we can start this transfert in more less 1 week ? (due to my availability) Or @gounthar do you want to follow-up this ?

[gounthar]

I can't perform the transfer either, as I'm not an org owner of jenkinsci. 🤷

[dduportal]

Hi,

Thanks for the update.

I personnally cannot perform the transfer because it must be initiated by an org owner of jenkinsci

Perhaps we can start this transfert in more less 1 week ? (due to my availability) Or @gounthar do you want to follow-up this ?

Ping @timja , is it ok for you to perform this transfer next week to help Valentin here?

[timja]

thats fine just ping when you want it done

[dduportal]

Ping @jonesbusy do you have a bit of time on this this week?

[jonesbusy]

Sure @dduportal I'm back from vaccation

@timja Can you perform the repository transfer from jenkinsci to jenkins-infra ?

Thanks

[timja]

I was able to transfer it although have no permissions on it, @dduportal can you setup standard permissions please? https://github.com/jenkins-infra/plugin-modernizer-tool

[jonesbusy]

and @gounthar @jonesbusy has maintainers

[dduportal]

Update:

• Permissions added (@jonesbusy you have an invitation from GitHub in your mailbox to join the jenkins-infra org.)
• I've opened a PR (https://github.com/jenkins-infra/plugin-modernizer-tool/pull/385) to update the references I could quickly find
• ci.jenkins.io configuration is currently being updated (Org Job Tools)

[dduportal]

Update:

• ci.jenkins.io has updated the job (I added a 2nd GH Org scanning for jenkins-infra, with the associated credential). Works well!
• Added a branch protection for ci.jio check on PRs: @jonesbusy let me know if it is ok, enough or too much?

[jonesbusy]

Thank you so much! Let me check but I think everything looks good!

[jonesbusy]

Something I didn't check and proabably broken is the cd artifactory token if they will still be generated by the RPU or not.

This is not critical, in any case we need to redo our release/packinging process to deliver the CLI

[dduportal]

Something I didn't check and proabably broken is the cd artifactory token if they will still be generated by the RPU or not.

This is not critical, in any case we need to redo our release/packinging process to deliver the CLI

Here you are ;) https://github.com/jenkins-infra/repository-permissions-updater/pull/4171

[dduportal]

Update: I've set up permissions (GitHub App and GitHub Organization Secrets) for the future "Binary and Homebrew Tap" capability:

You can use these secrets on the GHA (only on branches, not on PRs) workflow builds to publish homebrew taps.

Not: I recommend you to use the default GITHUB_TOKEN to publish Docker images and binary in the repository itself,

and use the secrets above with the https://github.com/tibdex/github-app-token GH Action to publish the homebrew tap.

[dduportal]

@jonesbusy @gounthar are there other items required by the Jenkins Infra team for driving this issue to completion?

[jonesbusy]

Any idea why the CI status is not reported here ? https://github.com/jenkins-infra/plugin-modernizer-tool/pull/383

Am I supposed to see the job here https://ci.jenkins.io/job/Infra/ ? Not sure where to locate the new job

[dduportal]

Am I supposed to see the job here https://ci.jenkins.io/job/Infra/ ? Not sure where to locate the new job

It should not have changed its location: https://ci.jenkins.io/job/Tools/job/plugin-modernizer-tool/

(as per https://github.com/jenkins-infra/helpdesk/issues/4262#issuecomment-2476240769)

[dduportal]

Any idea why the CI status is not reported here ? jenkins-infra/plugin-modernizer-tool#383

I'm not sure. Currently looking at Any idea why the CI status is not reported here ? https://github.com/jenkins-infra/plugin-modernizer-tool/pull/383 (in which the job ran but failed)

[dduportal]

Any idea why the CI status is not reported here ? jenkins-infra/plugin-modernizer-tool#383

I'm not sure. Currently looking at Any idea why the CI status is not reported here ? jenkins-infra/plugin-modernizer-tool#383 (in which the job ran but failed)

I see the following error in the pipeline output:

17:29:42  [GitHub Checks] Failed Publishing GitHub checks: org.kohsuke.github.HttpException: {"message":"Resource not accessible by integration","documentation_url":"https://docs.github.com/rest/checks/runs#create-a-check-run","status":"403"}
17:29:42  
17:29:42  Could not update commit status. Message: {"message":"Resource not accessible by integration","documentation_url":"https://docs.github.com/rest/commits/statuses#create-a-commit-status","status":"403"}

=> I guess there is a permission error, I need to diagnose a bit more deeply

[dduportal]

@jonesbusy problem solved for the checks:

• The GitHub App used in ci.jenkins.io to access the jenkins-infra repositories was missing this repository? I've updated it.
• Verified by triggerring a MB branch scan which did not fail
• Also triggered a manual build of https://github.com/jenkins-infra/plugin-modernizer-tool/pull/383 which reported the proper check

=> I also updated the branch protection to have the "Jenkins" check required (instead of the former 'pr-head' which can be confusing)