jena icon indicating copy to clipboard operation
jena copied to clipboard
verified publisher icon apache

Changelog

Open gaeljw opened this issue 3 years ago • 6 comments

Hello folks,

More a question than an issue but did I miss something or there's no changelog for the releases?

It's always a bit mysterious to upgrade dependencies when no changelog is provided. It would be nice if there was one (not necessarily on GitHub, on the official website is good as well).

Thanks :)

gaeljw avatar May 05 '22 07:05 gaeljw

Announcement: https://lists.apache.org/thread/qgjynztx70fhz7h86o6dfopv14dtgrc2 with headlines:

and the details:

GH issues:
https://github.com/apache/jena/issues?q=is%3Aissue+closed%3A2022-01-30..2022-05-04

JIRA Tickets this release:
https://s.apache.org/jena-4.5.0-jira

afs avatar May 05 '22 08:05 afs

Thanks for the quick answer @afs

Could be nice to move the CHANGELOG in the code itself so that it's not necessary to be registered on the mailing list to easily get it. It could also be retrieved by tools like RenovateBot or similar that automatically suggest dependencies upgrade.

I don't know how it fits in Apache practices though :shrug:

At least I know where to look for :pray:

gaeljw avatar May 05 '22 10:05 gaeljw

What other tools are there? Specifically, what does github use to populate the dependabot PR with change information?

afs avatar May 08 '22 14:05 afs

Not sure but looks like Dependabot looks for GitHub releases. And here comes the "each tool uses its own changelog" mess :(

gaeljw avatar May 08 '22 15:05 gaeljw

In Apache Commons the changelog is kept in the changes.xml file of the changes maven plugin. Then it's used to create the release notes with the changelog during the build.

Committers manually update it for every pull request that's not too small (e.g. fixed a comma in the javadoc). We also update for every dependabot pull request.

You eventually get used to this workflow, but it adds extra work on the developers. Maybe we could try something like a bash script to retrieve the changes from JIRA and GitHub issues/PRs, and then have a pre-release step to go through this list polishing/editing it… but that'd probably take a while too :disappointed_relieved:

kinow avatar May 08 '22 19:05 kinow

Jena has different audiences - the ANN has the main headlines and anything that users, including non-dev Fuseki users, might like to know about. We do include the list of changes in detail via searches of JIRA and now GH issues. And there are PRs for even more detail.

Adding more to the release process is a one way process - once we do it, we can't easily stop doing it.

afs avatar May 09 '22 13:05 afs

As a maintainer of a project with a Jena dependency, I expect to find a list of breaking changes (new behavior) in the release notes. The last release (4.7 0) had 259 commits changing 1,554 files and a very sparse release announcement 😅

https://github.com/apache/jena/compare/jena-4.6.1...jena-4.7.0

https://lists.apache.org/thread/2flk0xyz1bf0xpfmk1jgz74cvdog40z9

As for dependabot (which we also use), it may further be confused as the different components share the same repo and release tag. If pleasing dependabot is the goal, some testing should be done in a separate repo to confirm that release notes and/or a changelog is picked up for the individual components.

mr-c avatar Jan 04 '23 06:01 mr-c

@mr-c Which parts of Jena do you use?

afs avatar Jan 05 '23 13:01 afs

@afs, I think @mr-c saw the new version via this PR to cwlviewer: https://github.com/common-workflow-language/cwlviewer/pull/497

In CWLViewer there's a SpringBoot service bean that parses RDF and creates a model of the CWL workflow graph and also includes methods that fetch some data from the model via SPARQL queries like workflow inputs, outputs, etc: https://github.com/common-workflow-language/cwlviewer/blob/d5e1f5bfb85256e8a69441aaea62c01f398f43f9/src/main/java/org/commonwl/view/cwl/RDFService.java#L41-L43

kinow avatar Jan 05 '23 16:01 kinow

Strange - it uses DatasetAccessor which was removed several versions ago, and was deprecated for versions before that. jena-arq isn't in the dependencies, only jena-core. So a different version (3.9.0 is in an XML comment; 3.17.0 in dependabot.yml) must be coming via a transitive dependency.

afs avatar Jan 05 '23 18:01 afs

`mvn dependency:tree` 
[INFO] org.commonwl.view:cwlviewer:jar:1.4.6-SNAPSHOT
[INFO] +- org.springframework.boot:spring-boot-starter-thymeleaf:jar:2.6.7:compile
[INFO] |  +- org.springframework.boot:spring-boot-starter:jar:2.6.7:compile
[INFO] |  |  +- org.springframework.boot:spring-boot:jar:2.6.7:compile
[INFO] |  |  +- org.springframework.boot:spring-boot-autoconfigure:jar:2.6.7:compile
[INFO] |  |  +- org.springframework.boot:spring-boot-starter-logging:jar:2.6.7:compile
[INFO] |  |  |  +- ch.qos.logback:logback-classic:jar:1.2.11:compile
[INFO] |  |  |  |  \- ch.qos.logback:logback-core:jar:1.2.11:compile
[INFO] |  |  |  +- org.apache.logging.log4j:log4j-to-slf4j:jar:2.17.0:compile
[INFO] |  |  |  |  \- org.apache.logging.log4j:log4j-api:jar:2.17.0:compile
[INFO] |  |  |  \- org.slf4j:jul-to-slf4j:jar:1.7.36:compile
[INFO] |  |  \- jakarta.annotation:jakarta.annotation-api:jar:1.3.5:compile
[INFO] |  +- org.thymeleaf:thymeleaf-spring5:jar:3.0.15.RELEASE:compile
[INFO] |  |  \- org.thymeleaf:thymeleaf:jar:3.0.15.RELEASE:compile
[INFO] |  |     +- org.attoparser:attoparser:jar:2.0.5.RELEASE:compile
[INFO] |  |     \- org.unbescape:unbescape:jar:1.1.6.RELEASE:compile
[INFO] |  \- org.thymeleaf.extras:thymeleaf-extras-java8time:jar:3.0.4.RELEASE:compile
[INFO] +- org.springframework.boot:spring-boot-starter-web:jar:2.6.7:compile
[INFO] |  +- org.springframework.boot:spring-boot-starter-json:jar:2.6.7:compile
[INFO] |  |  +- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:2.13.2:compile
[INFO] |  |  +- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.13.2:compile
[INFO] |  |  \- com.fasterxml.jackson.module:jackson-module-parameter-names:jar:2.13.2:compile
[INFO] |  +- org.springframework.boot:spring-boot-starter-tomcat:jar:2.6.7:compile
[INFO] |  |  +- org.apache.tomcat.embed:tomcat-embed-core:jar:9.0.62:compile
[INFO] |  |  \- org.apache.tomcat.embed:tomcat-embed-websocket:jar:9.0.62:compile
[INFO] |  +- org.springframework:spring-web:jar:5.3.19:compile
[INFO] |  \- org.springframework:spring-webmvc:jar:5.3.19:compile
[INFO] |     +- org.springframework:spring-aop:jar:5.3.19:compile
[INFO] |     +- org.springframework:spring-context:jar:5.3.19:compile
[INFO] |     \- org.springframework:spring-expression:jar:5.3.19:compile
[INFO] +- org.springframework.boot:spring-boot-starter-data-jpa:jar:2.6.7:compile
[INFO] |  +- org.springframework.boot:spring-boot-starter-aop:jar:2.6.7:compile
[INFO] |  |  \- org.aspectj:aspectjweaver:jar:1.9.7:compile
[INFO] |  +- org.springframework.boot:spring-boot-starter-jdbc:jar:2.6.7:compile
[INFO] |  |  +- com.zaxxer:HikariCP:jar:4.0.3:compile
[INFO] |  |  \- org.springframework:spring-jdbc:jar:5.3.19:compile
[INFO] |  +- jakarta.transaction:jakarta.transaction-api:jar:1.3.3:compile
[INFO] |  +- jakarta.persistence:jakarta.persistence-api:jar:2.2.3:compile
[INFO] |  +- org.hibernate:hibernate-core:jar:5.6.8.Final:compile
[INFO] |  |  +- org.jboss.logging:jboss-logging:jar:3.4.3.Final:compile
[INFO] |  |  +- net.bytebuddy:byte-buddy:jar:1.11.22:compile
[INFO] |  |  +- antlr:antlr:jar:2.7.7:compile
[INFO] |  |  +- org.jboss:jandex:jar:2.4.2.Final:compile
[INFO] |  |  +- com.fasterxml:classmate:jar:1.5.1:compile
[INFO] |  |  +- org.hibernate.common:hibernate-commons-annotations:jar:5.1.2.Final:compile
[INFO] |  |  \- org.glassfish.jaxb:jaxb-runtime:jar:2.3.6:compile
[INFO] |  |     +- org.glassfish.jaxb:txw2:jar:2.3.6:compile
[INFO] |  |     +- com.sun.istack:istack-commons-runtime:jar:3.0.12:compile
[INFO] |  |     \- com.sun.activation:jakarta.activation:jar:1.2.2:runtime
[INFO] |  +- org.springframework.data:spring-data-jpa:jar:2.6.4:compile
[INFO] |  |  +- org.springframework:spring-orm:jar:5.3.19:compile
[INFO] |  |  \- org.springframework:spring-tx:jar:5.3.19:compile
[INFO] |  \- org.springframework:spring-aspects:jar:5.3.19:compile
[INFO] +- org.springframework.data:spring-data-commons:jar:2.7.6:compile
[INFO] |  +- org.springframework:spring-core:jar:5.3.19:compile
[INFO] |  |  \- org.springframework:spring-jcl:jar:5.3.19:compile
[INFO] |  +- org.springframework:spring-beans:jar:5.3.19:compile
[INFO] |  \- org.slf4j:slf4j-api:jar:1.7.36:compile
[INFO] +- com.vladmihalcea:hibernate-types-55:jar:2.21.1:compile
[INFO] +- org.liquibase:liquibase-core:jar:4.18.0:compile
[INFO] |  +- javax.xml.bind:jaxb-api:jar:2.3.1:compile
[INFO] |  |  \- javax.activation:javax.activation-api:jar:1.2.0:compile
[INFO] |  +- org.yaml:snakeyaml:jar:1.29:compile
[INFO] |  \- com.opencsv:opencsv:jar:5.7.1:compile
[INFO] |     +- org.apache.commons:commons-text:jar:1.10.0:compile
[INFO] |     \- org.apache.commons:commons-collections4:jar:4.4:compile
[INFO] +- org.postgresql:postgresql:jar:42.3.4:runtime
[INFO] |  \- org.checkerframework:checker-qual:jar:3.5.0:runtime
[INFO] +- org.snakeyaml:snakeyaml-engine:jar:2.5:compile
[INFO] +- com.github.jabbalaci:graphviz-java-api:jar:f9bf94896776de6e98b5004e819b63fb4c15b15d:compile
[INFO] +- org.apache.taverna.language:taverna-robundle:jar:0.16.0-incubating-SNAPSHOT:compile
[INFO] |  +- org.apache.taverna.language:taverna-ro-vocabs:jar:0.16.0-incubating-SNAPSHOT:compile
[INFO] |  |  \- org.apache.jena:jena-arq:jar:3.7.0:compile
[INFO] |  +- com.fasterxml.jackson.core:jackson-annotations:jar:2.13.2:compile
[INFO] |  +- com.fasterxml.jackson.core:jackson-databind:jar:2.13.2.1:compile
[INFO] |  +- commons-configuration:commons-configuration:jar:1.10:compile
[INFO] |  |  +- commons-lang:commons-lang:jar:2.6:compile
[INFO] |  |  \- commons-logging:commons-logging:jar:1.1.1:compile
[INFO] |  +- org.apache.jena:jena-osgi:jar:3.7.0:compile
[INFO] |  |  +- org.apache.commons:commons-csv:jar:1.5:compile
[INFO] |  |  +- org.apache.thrift:libthrift:jar:0.10.0:compile
[INFO] |  |  \- org.osgi:org.osgi.core:jar:5.0.0:compile
[INFO] |  +- org.apache.servicemix.bundles:org.apache.servicemix.bundles.xerces:jar:2.11.0_1:compile
[INFO] |  |  \- xml-apis:xml-apis:jar:1.4.01:compile
[INFO] |  +- com.github.andrewoma.dexx:collection:jar:0.6:compile
[INFO] |  +- com.sun.xml.bind:jaxb-osgi:jar:2.3.0:compile
[INFO] |  \- javax.activation:activation:jar:1.1.1:compile
[INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.14.1:compile
[INFO] +- org.apache.jena:jena-core:jar:4.7.0:compile
[INFO] |  +- org.apache.jena:jena-base:jar:4.7.0:compile
[INFO] |  |  +- org.apache.jena:jena-shaded-guava:jar:4.7.0:compile
[INFO] |  |  \- commons-codec:commons-codec:jar:1.15:compile
[INFO] |  +- org.apache.jena:jena-iri:jar:4.7.0:compile
[INFO] |  \- commons-cli:commons-cli:jar:1.5.0:compile
[INFO] +- com.github.jsonld-java:jsonld-java:jar:0.13.4:compile
[INFO] |  +- org.apache.httpcomponents:httpclient-osgi:jar:4.5.13:compile
[INFO] |  |  +- org.apache.httpcomponents:httpclient:jar:4.5.13:compile
[INFO] |  |  +- org.apache.httpcomponents:httpmime:jar:4.5.13:compile
[INFO] |  |  +- org.apache.httpcomponents:httpclient-cache:jar:4.5.13:compile
[INFO] |  |  \- org.apache.httpcomponents:fluent-hc:jar:4.5.13:compile
[INFO] |  +- org.apache.httpcomponents:httpcore-osgi:jar:4.4.14:compile
[INFO] |  |  +- org.apache.httpcomponents:httpcore:jar:4.4.15:compile
[INFO] |  |  \- org.apache.httpcomponents:httpcore-nio:jar:4.4.15:compile
[INFO] |  \- org.slf4j:jcl-over-slf4j:jar:1.7.36:compile
[INFO] +- org.eclipse.jgit:org.eclipse.jgit:jar:6.1.0.202203080745-r:compile
[INFO] |  \- com.googlecode.javaewah:JavaEWAH:jar:1.1.13:compile
[INFO] +- org.apache.commons:commons-compress:jar:1.22:compile
[INFO] +- commons-io:commons-io:jar:2.11.0:compile
[INFO] +- org.springframework.boot:spring-boot-starter-validation:jar:2.6.7:compile
[INFO] |  +- org.apache.tomcat.embed:tomcat-embed-el:jar:9.0.62:compile
[INFO] |  \- org.hibernate.validator:hibernate-validator:jar:6.2.3.Final:compile
[INFO] |     \- jakarta.validation:jakarta.validation-api:jar:2.0.2:compile
[INFO] +- com.github.tbouron:spdx-license-checker:jar:1.0.0:compile
[INFO] |  \- org.apache.commons:commons-lang3:jar:3.12.0:compile
[INFO] +- org.springframework.boot:spring-boot-starter-test:jar:2.6.7:test
[INFO] |  +- org.springframework.boot:spring-boot-test:jar:2.6.7:test
[INFO] |  +- org.springframework.boot:spring-boot-test-autoconfigure:jar:2.6.7:test
[INFO] |  +- com.jayway.jsonpath:json-path:jar:2.6.0:test
[INFO] |  |  \- net.minidev:json-smart:jar:2.4.8:test
[INFO] |  |     \- net.minidev:accessors-smart:jar:2.4.8:test
[INFO] |  |        \- org.ow2.asm:asm:jar:9.1:test
[INFO] |  +- jakarta.xml.bind:jakarta.xml.bind-api:jar:2.3.3:compile
[INFO] |  |  \- jakarta.activation:jakarta.activation-api:jar:1.2.2:compile
[INFO] |  +- org.assertj:assertj-core:jar:3.21.0:test
[INFO] |  +- org.hamcrest:hamcrest:jar:2.2:test
[INFO] |  +- org.junit.jupiter:junit-jupiter:jar:5.8.2:test
[INFO] |  |  +- org.junit.jupiter:junit-jupiter-params:jar:5.8.2:test
[INFO] |  |  \- org.junit.jupiter:junit-jupiter-engine:jar:5.8.2:test
[INFO] |  |     \- org.junit.platform:junit-platform-engine:jar:1.8.2:test
[INFO] |  +- org.mockito:mockito-core:jar:4.0.0:test
[INFO] |  |  +- net.bytebuddy:byte-buddy-agent:jar:1.11.22:test
[INFO] |  |  \- org.objenesis:objenesis:jar:3.2:test
[INFO] |  +- org.mockito:mockito-junit-jupiter:jar:4.0.0:test
[INFO] |  +- org.skyscreamer:jsonassert:jar:1.5.0:test
[INFO] |  |  \- com.vaadin.external.google:android-json:jar:0.0.20131108.vaadin1:test
[INFO] |  +- org.springframework:spring-test:jar:5.3.19:test
[INFO] |  \- org.xmlunit:xmlunit-core:jar:2.8.4:test
[INFO] +- org.testcontainers:junit-jupiter:jar:1.17.6:test
[INFO] |  +- org.testcontainers:testcontainers:jar:1.17.6:test
[INFO] |  |  +- junit:junit:jar:4.13.2:test
[INFO] |  |  |  \- org.hamcrest:hamcrest-core:jar:2.2:test
[INFO] |  |  +- org.rnorth.duct-tape:duct-tape:jar:1.0.8:test
[INFO] |  |  |  \- org.jetbrains:annotations:jar:17.0.0:test
[INFO] |  |  +- com.github.docker-java:docker-java-api:jar:3.2.13:test
[INFO] |  |  \- com.github.docker-java:docker-java-transport-zerodep:jar:3.2.13:test
[INFO] |  |     +- com.github.docker-java:docker-java-transport:jar:3.2.13:test
[INFO] |  |     \- net.java.dev.jna:jna:jar:5.8.0:test
[INFO] |  \- org.junit.jupiter:junit-jupiter-api:jar:5.8.2:test
[INFO] |     +- org.opentest4j:opentest4j:jar:1.2.0:test
[INFO] |     +- org.junit.platform:junit-platform-commons:jar:1.8.2:test
[INFO] |     \- org.apiguardian:apiguardian-api:jar:1.1.2:test
[INFO] +- org.testcontainers:postgresql:jar:1.17.6:test
[INFO] |  \- org.testcontainers:jdbc:jar:1.17.6:test
[INFO] |     \- org.testcontainers:database-commons:jar:1.17.6:test
[INFO] \- org.mockito:mockito-inline:jar:4.11.0:test

It's due to Taverna:

[INFO] +- org.apache.taverna.language:taverna-robundle:jar:0.16.0-incubating-SNAPSHOT:compile
[INFO] |  +- org.apache.taverna.language:taverna-ro-vocabs:jar:0.16.0-incubating-SNAPSHOT:compile
[INFO] |  |  \- org.apache.jena:jena-arq:jar:3.7.0:compile

https://github.com/common-workflow-language/cwlviewer/issues/498

Thanks @afs!

kinow avatar Jan 05 '23 20:01 kinow

That anything works 3.7.0/4.7.0 is a pleasant (and unsupported!) surprise.

afs avatar Jan 05 '23 20:01 afs

Apache Taverna retired.

The codebase source is archived at 0.16.0-incubating-SNAPSHOT. There are jars in Apache snapshot repository but I'm not sure they supposed to still be there. They may be deleted (e.g. in a diskspace clearup).

https://github.com/common-workflow-language/cwlviewer/blob/main/pom.xml#L43

afs avatar Jan 05 '23 20:01 afs

Not sure if CWLViewer is using Jena 4 and 3 at the same time. Or if it's using Taverna for the research objects files, and Jena4 graphs for the CWL workflow (I hope that's what it's doing). I've created an issue to sort out these two versions of Jena in the CWL Viewer later (it has other things in the pipeline, Spring Data 3 upgrade, newer versions of hibernate/postgres, migration of servers, etc).

@afs coming back to changelog in Jena;

Could be nice to move the CHANGELOG in the code itself so that it's not necessary to be registered on the mailing list to easily get it. It could also be retrieved by tools like RenovateBot or similar that automatically suggest dependencies upgrade.

What other tools are there? Specifically, what does github use to populate the dependabot PR with change information?

As far as I know, it parses the changelog from files in the source code repo. I knew it used CHANGELOG. I looked at their docs but it was not clear, so had a look at their source code. This is the list of files they appear to accept:

In each of those files, it uses a "pruner" to locate the desired version: https://github.com/dependabot/dependabot-core/blob/0603ec94756a975b2ea928acf7a74213198599be/common/lib/dependabot/metadata_finders/base/changelog_finder.rb#L124

That's what I could understand. So I think having a CHANGELOG.md file in the root of the Jena repository, neighbor to README and LICENSE, would include our changelog in dependabot. For non-dependabot users, I think most people are familiar with, and look for these files.

On how to generate these files;

I think we could try to automate it if it's really hard to keep track of the changes, maybe extract it from Git and/or JIRA. But I think it could be simpler to decide what to include when merging pull requests.

We could update our pull request template to include a line like:

  • CHANGES.md entry included if this is a change that can affect users

To remind us whether we should include something in the changelog. This way, main would always have the snapshot changelog, and during the release we would have to remember to update the changelog entry (date/version). And after the release start a new changelog for the next version…

Not sure if that would complicate much the release process.

kinow avatar Jan 06 '23 21:01 kinow

Not sure if CWLViewer is using Jena 4 and 3 at the same time.

Jena uses ServiceLoader to initialize so if jars from both on the classpath, then, yes, it is both (jena-core from v4, RIOT from v3). Since 3.0.1, RIOT plugged itself into jena-core as an RDFReader.

afs avatar Jan 06 '23 21:01 afs

Done with https://github.com/apache/jena/blob/main/CHANGES.txt

afs avatar Oct 22 '23 11:10 afs