Source changelogs for programmatic consumption

[gioamato]

It might be useful to source changelogs from WordPress.org with version tags (eg. dependabot could show it in PRs).

I could try to add it and open a PR if you just point me in the right direction.

[NoelLH]

Hey @gioamato, thanks for the suggestion. I can see how this might be useful to expose somewhere in the frontend – I'm not sure if there is a working Dependabot use case per se as yet, but let me know if that's wrong?

I think it's potentially a reasonably big change. If you are hoping to consistently pull out the exact changes for each version number I'm also not sure that's going to work. I skimmed some entries from the API and the formatting is pretty inconsistent. In each plugin in the array returned by API v1.1, sections.changelog is a big blob of HTML and sometimes has a paragraph prefix before the versions, sometimes not.

While the version numbered bits later in (or sometimes from the start of) the changelog seem to consistently show in <h4>s their format is not consistent and sometimes includes stuff like date suffixes, which diverges from the Subversion repository's version numbers format. So I suspect these may be impossible to reliably reconcile with the internal version numbers used for Composer dependency management.

If you do still want to look at saving the whole changelog at package level, I'd suggest the steps would be something like:

1. Add a model field for the info, probably in Outlandish\Wpackagist\Entity\Package.
2. Add a database migration to get this safely added to schemas: bin/console doctrine:migrations:diff may help here.
3. Expand the use of the rarst/wporg-client to request the sections key I mentioned above or just sections.changelog if possible, alongside version metadata.
4. On updates, save that HTML against the field you added – possibly using strip_tags() or doing more involved preprocessing, depending on your plan for outputting the data.
5. If you want to output it within the Wpackagist web app or anywhere else that might end up in a web context, sanitise the HTML to avoid security risks when doing so. If you are targeting non-web output you will need to strip the HTML and probably do some other reformatting to make these legible as plain text.

[chrillep]

+1 on this same for renovate. not sure about the parsing tho. however github supports html.

example urls

• https://api.wordpress.org/plugins/info/1.2/?action=plugin_information&slug=wordpress-seo
• https://api.wordpress.org/plugins/info/1.2/?action=plugin_information&slug=contact-form-7
• https://api.wordpress.org/plugins/info/1.2/?action=plugin_information&slug=redirection

fetch("https://api.wordpress.org/plugins/info/1.2/?action=plugin_information&slug=redirection")
  .then(response => response.json())
  .then(data => console.log(data.sections.changelog))
  .catch(error => console.log(error))

refs

• https://github.com/roots/wordpress/issues/47
• https://dd32.id.au/projects/wordpressorg-plugin-information-api-docs/

[carstingaxion]

Wanted to ask for this, too. Thank you.

[chrillep]

+1 on this same for renovate. not sure about the parsing tho. however github supports html.

example urls

• https://api.wordpress.org/plugins/info/1.2/?action=plugin_information&slug=wordpress-seo
• https://api.wordpress.org/plugins/info/1.2/?action=plugin_information&slug=contact-form-7
• https://api.wordpress.org/plugins/info/1.2/?action=plugin_information&slug=redirection

fetch("https://api.wordpress.org/plugins/info/1.2/?action=plugin_information&slug=redirection")
  .then(response => response.json())
  .then(data => console.log(data.sections.changelog))
  .catch(error => console.log(error))

refs

• Add changelog from WordPress to tag/release roots/wordpress#47
• https://dd32.id.au/projects/wordpressorg-plugin-information-api-docs/

@LeoColomb id like to work on this, but before i start, id like your takeaways. Im tagging you since youve done this once, but for roots/wordpress.

[chrillep]

hmm maybe its better if renovate and dependabot fix this instead.

• https://github.com/renovatebot/renovate/discussions/24072
• https://github.com/renovatebot/renovate/issues/24105

[LeoColomb]

Im tagging you since youve done this once, but for roots/wordpress.

Well, I'm not sure the process will be the same, as this plugins API provides an (almost) well formatted and well-structured changelog value. You should feel lucky 😁

My main takeaways though are that WordPress changelog is:

• Absolutely not ready for API usage. Scraping is never reassuring as the structure is not guarantied.
• Published about 30-35 min after releases API is updated, while managed Renovate/Dependabot do their stuff about 20-30 min after a release is published. Enjoy the relaxing delay 😢

hmm maybe its better if renovate and dependabot fix this instead.

Probably better, I agree, but comes with some additional challenges (especially HTML handling). I'd also note plugins with type:wordpress-plugin can be published at/retrieved from many different places, so the script should expect failures and handle them gracefully.