Extensions index is missing many implementations

[jtnord]

Describe your use-case which is not covered by existing documentation.

https://www.jenkins.io/doc/developer/extensions is supposed to list known ExtensionPoints and the known (ie public jenkins) implemtnations via an Extension.

However it seems like there are missing implementations

For example https://www.jenkins.io/doc/developer/extensions/jenkins-core/#queuetaskdispatcher

hudson.model.queue.QueueTaskDispatcher is implemented by branch-api plugins RateLimitBranchProperty.QueueTaskDispatcherImpl but it is not listed.

Reference any relevant documentation, other materials or issues/pull requests that can be used for inspiration.

No response

[MarkEWaite]

Sorry about that. No progress on the issue since you reported it almost 3 months ago as https://github.com/jenkins-infra/backend-extension-indexer/issues/58

I'll try to spend some time on it today as part of Docs Office Hours preparation. Hopefully a first attempt to rollback to a working version will be successful, then later I can do more investigation to understand what broke the indexer.

[jtnord]

Thanks MarK.

I thought I filed it before and then could not find it - somewhat obviously as I was looking in the wrong place!

[MarkEWaite]

Thanks MarK.

I thought I filed it before and then could not find it - somewhat obviously as I was looking in the wrong place!

Apologies for the non-progress. I hope a short term workaround will be available by some form of rollback with a long term solution being to add tests to the backend extension indexer to detect whatever condition caused the failure.

[lemeurherve]

I thought I filed it before and then could not find it - somewhat obviously as I was looking in the wrong place!

This is the third help desk issue on this topic already 😅 The others have been moved since. (Note: we shouldn't transfer this one)

The other one for ref, linked in https://github.com/jenkins-infra/backend-extension-indexer/issues/58: https://github.com/jenkins-infra/backend-extension-indexer/issues/42

[KalleOlaviNiemitalo]

Also https://github.com/jenkins-infra/jenkins.io/issues/6406

[basil]

Caused by https://github.com/jenkinsci/plugin-pom/pull/100 — any plugin that has been incrementalified uses flatten-maven-plugin, which removes the Jenkins core dependency from the flattened POM and breaks the logic in jenkins-infra/backend-extension-indexer, which relies on the Jenkins core dependency being on the classpath in order to compile the plugin and list its extensions. Suggest updating the flatten-maven-plugin configuration to retain the Jenkins core dependency or updating the logic in jenkins-infra/backend-extension-indexer to re-add the Jenkins core dependency to the classpath, if not rewriting jenkins-infra/backend-extension-indexer from scratch.

[jglick]

compile the plugin

:exclamation: Whatever this was doing, it sounds like it needs to be rewritten. Nothing should need to be compiled. Take the *.hpi from the UC, look for META-INF/annotations/hudson.Extension + META-INF/annotations/org.jenkinsci.plugins.variant.OptionalExtension (or the *.txt variants produced by the plugin toolchain since https://github.com/jglick/sezpoz/commit/adf2093128f4bbcc8fadefdcc65d2b168964913e), and you have the basic information. GH links you can guess at, these are best-effort anyway. Descriptions can be replaced with links to https://javadoc.jenkins.io/.

[basil]

Whatever this was doing, it sounds like it needs to be rewritten.

Hence my previous suggestion in https://github.com/jenkins-infra/helpdesk/issues/3746#issuecomment-1718138983:

Suggest […] rewriting jenkins-infra/backend-extension-indexer from scratch

[basil]

@jglick Since this regression was caused by https://github.com/jenkinsci/plugin-pom/pull/100, should https://github.com/jenkins-infra/backend-extension-indexer/issues/58 be assigned to you?

[zbynek]

Since this came up multiple times and the proposed fix of rewriting the extension indexer sounds like a long term projet, would it make sense to find some interim solution ? Maybe a disclaimer in the extension point index linking to https://web.archive.org/web/20210308005332/http://www.jenkins.io/doc/developer/extensions/ explaining that some content went missing 🤷

[basil]

would it make sense to find some interim solution ?

I suggested an interim solution in https://github.com/jenkins-infra/helpdesk/issues/3746#issuecomment-1718138983:

updating the logic in jenkins-infra/backend-extension-indexer to re-add the Jenkins core dependency to the classpath

[jglick]

If that works, it sounds like a good hotfix for this symptom. The fundamental mistake is the attempt to use a deployed POM to build sources, which is not what deployed POMs are for (Maven 4 is supposed to enforce the distinction), and of course relying on sources rather than binaries is inherently fragile (https://github.com/jenkins-infra/backend-extension-indexer/issues/42 sounds like another consequence).

[daniel-beck]

I suggested an interim solution in #3746 (comment):

updating the logic in jenkins-infra/backend-extension-indexer to re-add the Jenkins core dependency to the classpath

That's basically what I did in https://github.com/jenkins-infra/backend-extension-indexer/pull/95

The fundamental mistake is the attempt to use a deployed POM to build sources, which is not what deployed POMs are for

What are they for?

FWIW neither that file, nor the source jar contain any reference to the core dependency. That seems wrong.

[jglick]

What are they for?

Resolving dependencies from Maven builds.

relying on sources rather than binaries is inherently fragile

Not sure if this is the cause of https://github.com/jenkins-infra/backend-extension-indexer/issues/97, but note that there are some extension implementations in that plugin which are currently defined in Groovy sources, so they could not possibly be detected by parsing (Java) source files. You really need to inspect bytecode and bytecode alone.

[daniel-beck]

Resolving dependencies from Maven builds.

Wouldn't that mean that the core dependency should be there?

[daniel-beck]

they could not possibly be detected by parsing (Java) source files. You really need to inspect bytecode and bytecode alone.

This is probably a good path forward. We'll lose the Javadoc excerpts, but since we're linking to javadoc.jenkins.io it should be fine. I intend to take a stab at this when I have some time, but if someone else wants to do it, feel free.