Update to proto 4.X

[rafael-andrade-3010]

Hello! I would like to check with you if you have plans to update this lib to use protoc 4.X, I tried a bit locally but no success.

Thanks in advance

[RomanIakovlev]

Hi! From what I understand, protoc as such has current version of 26.X, while protobuf for Java have current version 4.26.x, or some such. Having said that, when you suggest upgrading to protoc 4.X you probably mean protobuf 4.X, right? Because protoc version range is already in twenties.

I wonder if you have tried bumping the version here? https://github.com/RomanIakovlev/timeshape/blob/70079536dcc3f1a3df3d112b4efb75cd47b285c8/build.sbt#L82

If that doesn't work right away, maybe upgrading the sbt-protoc plugin and "com.thesamet.scalapb" %% "compilerplugin" library in https://github.com/RomanIakovlev/timeshape/blob/master/project/plugins.sbt first would help?

Putting those technicalities aside, I think there's also a question of backwards compatibility here. Increasing the major version from 3 to 4 is a breaking change, so people who still rely on major version 3 of protobuf won't be able to use the new releases of Timeshape if it switches to major version 4. I've just realized the version 3.21.12 that's currently used is quite ancient, so it would definitely make sense to upgrade to 3.25.x, but going directly to major version 4 is probably too early.

Speaking of when we could switch to protobuf major version 4, I would follow this version support guide here: https://protobuf.dev/support/version-support/#java:

That is, I'd upgrade to major version 4 somewhere towards the end of this year, or in early 2025, when the public support ends.

To get support for major version 4 earlier, it will be necessary to build a new artifact, e.g. net.iakovlev:timeshape-pb4:x, or something like that, and publish it in parallel with the standard net.iakovlev:timeshape:x. In all honesty I'm not too keen on doing that myself, because that would complicate the build definition somewhat, and I personally don't have a need for that.

If you need it, the easiest way would probably be for you to (temporarily) fork Timeshape, build that -pb4 version for yourself, and use it until the main version switches to the protobuf major 4 version.

[AeroSecGeek]

Just a head's up call: there was a vulnerability detected in the protobuf library, for details you can visit: https://nvd.nist.gov/vuln/detail/CVE-2024-7254 Unfortunately, at the moment there is no fix available but I think it would be great to update the library as soon as a fix is available.

[danio]

protobuf-jav 3.25.5 was released 18th Sept which addresses CVE-2024-7254

I think it's worth upgrading to that at least @RomanIakovlev

[RomanIakovlev]

Thanks for the heads up @AeroSecGeek and @danio! An upgrade in the 3.x series is very much uncontroversial, so feel free to send a PR and I'll merge it and make a release.

[RomanIakovlev]

I've just released Timeshape version 2024a.23 with protobuf 3.25.5. Thanks @AeroSecGeek for your contribution!

[danio]

great news, thank you both!

[AeroSecGeek]

@RomanIakovlev I think you need to do also a new release of net.iakovlev.geojson-proto since the protobuf library is used in there?

[RomanIakovlev]

@AeroSecGeek yes, you're absolutely right, I've overlooked it. This means, a new release of Timeshape will be needed as well, because it's going to depend on a new build of geojson-proto.

[RomanIakovlev]

Ok it's done properly now, Timeshape 2024a.24 is released with geojson-proto 1.1.4 (which has upgraded protobuf).

[AeroSecGeek]

@RomanIakovlev Thank you very much, highly appreciated!

[internetstaff]

It's late 2024, any chance of an upgrade? :)

[ellabi]

Hi, any plans to use protobuf 4.X.X?

[RomanIakovlev]

I understand the importance of this upgrade, but I'm currently unable to spend enough time on this. If someone sends a PR, I'll gladly merge it and cut a release. 👍

[internetstaff]

Curious: why sbt? Seems like reverting to Maven would encourage more contributions.

[RomanIakovlev]

Curious: why sbt? Seems like reverting to Maven would encourage more contributions.

Agree, if I were making this choice nowadays I'd go with Maven. I'm open to switching to Maven. There's a bunch of logic written in sbt now related to fetching timezone data and building the core artifact, which Maven cannot really do well. It has to be replaced with some shell scripting probably, so it's quite a bit of an effort.

[internetstaff]

Any hope on this one? Seems like it's close. I submitted a new PR a couple weeks ago ...

thanks!

[RomanIakovlev]

@internetstaff Thanks for the PR and the ping, I'me missed the PR notification somehow. I'll take a look!

[RomanIakovlev]

@internetstaff I've merged your PR, but there seems to be a problem with publishing to Maven Central. It's the first time I try to publish after migrating to Maven from sbt, so this is a bit as expected. I've tried several fixes, but nothing seems to work yet.

I don't have a lot of time to dedicate to this now. If anyone could figure out what's wrong, or a workaround, that would be appreciated. One thing to mention is that the process of publishing to Maven Central has changed since the previous release, adding a new moving part here, unfortunately.

[internetstaff]

@RomanIakovlev It looks like it worked, except geojson was published as a SNAPSHOT release, so it is not available on central.

I'm unclear why that artifact uses a different versioning system.

[RomanIakovlev]

Regrading geojson-proto being a snapshot, this is by design (from the sbt times) but is probably going to change in the future. I wouldn't be too concerned about it, because the main artifact depends on the previous non-snapshot version.

The problem now is that the artifact is still not promoted to the Maven Central for some reason. Only 2025b.26 is available on https://search.maven.org/artifact/net.iakovlev/timeshape/. Maybe it's me misunderstanding what's going on, or missing some configuration in the https://central.sonatype.com. If you have ideas, I'm all ears.

P.S. it's available here https://central.sonatype.com/artifact/net.iakovlev/timeshape/2025b.27, but I'm not sure if it's the new default location for Maven artifacts?

[internetstaff]

2025b.27 shows up for me fine, and I'm able to use it.

Unfortunately, geojson-proto does not show up, as Central will not post SNAPSHOT releases - so 2025b.27 is broken.

[RomanIakovlev]

Oh I see, the current main Timeshape release refers to the SNAPSHOT version of geojson-proto. This is definitely unexpected and wrong, it should refer to a stable version.

The release process for geojson-proto used to be like this:

1. Make change in geojson-proto, e.g. bump the protobuf version
2. Make new non-snapshot release of geojson-proto
3. Update main Timeshape artifact to depend on new non-snapshot version of geojson-proto
4. Make new release of main Timeshape artifact

So we probably have to do something similar here as well.

[internetstaff]

Have you considered syncing the geojson-proto version with the parent version?

I understand why the version semantics are different, but it would simplify things. It does not seem like geojson-proto is referenced much independently on maven central.

It seems like you are manually setting versions before a release. You could use e.g. <version>${project.parent.version}</version> to simplify that a bit.

[RomanIakovlev]

I think there's another way. Looking at this new Sonatype Central publishing plugin, it seems there's a setting to not re-publish the existing artifacts. If it works as advertised, we can set it and stop doing all these SNAPSHOT shenanigans with geojson-proto, just set its version to the non-SNAPSHOT and let the plugin deal with it, because previously using the SNAPSHOT mechanism was used to avoid precisely these errors with re-publishing non-SNAPSHOT versions.

I'm not too keen on using the same version for geojson-proto and the main artifact, mainly because the main artifact has non-standard versioning scheme, while geojson-proto follows the semver standard. It's not too big of a deal, because it's probably not used by anyone than Timeshape, but I like to keep things nice and clean, even if no one is watching.

[juarezr]

Upgrade from 3.25.x to 4.x

It looks like the end of support for protobuf-java 3.25.X series didn't go as planned. Too many projects are still depending on it at this moment.

Below is the repost from #174 for adding to the discussion.

Reposting

I also noticed that geojson:1.1.6 was upgraded to depend on protobuf-java:4.31.1 from the previous 3.25.x version.

As we have to reconcile the use of proto in timeshape with the Apache Spark, Dataproc and also our internal project use, I tried to keep the minimum denominator among the versions.

So, I have some questions about the upgrade:

• It would be possible to keep using the runtime version 3.25.x of protobuf-java?
• Or maybe keep just the runtime compatibility between the 3.25.x and 4.2+ versions.
• It looks like the 3.25.x versions are already supported:
  • The least supported version is defined by the google-cloud-java that is the 3.19.4.
  • The Google's java-cloud-bom indicates that they support a libraries-bom-protobuf3 besides the 4.26+ ones.

[internetstaff]

@RomanIakovlev Do you intend to try that mechanism? I'm not sure I can add much, since it's just setting the version and running a release. Seems like you will need to increment the main version also, since 27 is broken, and employing that setting will also not update it?

[internetstaff]

Any luck? :)

[RomanIakovlev]

Looks like both geojson-proto 1.1.6 and timeshape 2025b.28 using geojson-proto 1.1.6 were successfully published just now. Could you give it a try and let me know if it works for you?

[internetstaff]

Looks good to me. Everything seems to work, and with proto 4.x.

thanks!