Rethink how release notes are produced

[axw]

At the moment when we create changes with user-visible impacts, we add an entry to a changelog: https://github.com/elastic/apm-server/blob/master/changelogs/head.asciidoc. This becomes a pain when backporting PRs as not all entries in changelogs/head.asciidoc will or should be carried over to older branches, and so this file is a constant source of merge conflicts. This makes automated backports impractical in many cases.

I would like us to consider alternatives, such as:

• Kibana's approach of using labels (https://github.com/elastic/kibana/tree/master/packages/kbn-docs-utils/src/release_notes)
• Elasticsearch's proposed approach of having fine-grained changelog files (https://github.com/elastic/elasticsearch/issues/67335)
• Conventional commits

[axw]

@bmorelli25 do you have any opinions on the current approach, and anything we could do to improve the docs at the same time?

@gtback in https://github.com/elastic/elasticsearch/issues/67335#issuecomment-759078203 you mentioned you're looking into standardising release notes/changelogs across products. Any updates on that?

[bmorelli25]

@graphaelli and I chatted briefly about this last month. There's a neat effort going on right now, dubbed elastic.co/new. The idea is to standardize and centralize all Elastic Stack release notes. You can read the spec here: https://github.com/elastic/elastic-docs/issues/254 (sorry to anyone outside of the Elastic org, this is an internal link).

The ES issue you link above is related to this effort. Greg is in that working group, so hopefully he can chime in with additional interesting information.

[gtback]

@gtback in elastic/elasticsearch#67335 (comment) you mentioned you're looking into standardising release notes/changelogs across products. Any updates on that?

I don't have many updates on this. The plan is still to have a basic JSON format that each team can produce however they want, that gets picked up by some process and indexed into the same App Search engine. The closest thing to a "spec" is here but it's definitely still a work in progress. Happy to chat more about this with anyone who's interested.

[simitt]

Throwing another option into the mix that @endorama recently presented: https://github.com/elastic/elastic-agent-changelog-tool.