protobufs icon indicating copy to clipboard operation
protobufs copied to clipboard

GA "Push commit to schema registry" is failing since october

Open KomelT opened this issue 10 months ago • 10 comments

GA "Push commit to schema registry" has been failing since October and new protobuf versions aren't published.

Image

KomelT avatar Jan 20 '25 21:01 KomelT

We can stop publishing to bsr as we aren't using any code generated from there anymore.

sachaw avatar Jan 21 '25 13:01 sachaw

Oh, I see. I thought Web UI was using protobuf definitions from there, but it inherits them from @meshtastic/js. The JS library gets protobuf definitions from https://jsr.io/@meshtastic/protobufs.

I hope I am not wrong again.

KomelT avatar Jan 21 '25 19:01 KomelT

Yeah, the protobuf and main library is being published to JSR A major advantage of that is having a less complicated .npmrc setup for projects that consume the protobufs as a direct or indirect dependency

sachaw avatar Jan 22 '25 01:01 sachaw

Ha, interesting. The JSR looks like it is platform-specific (JS and TS). Buf supports Swift, ts, js, python, go, c, rust, scala, and more. Isn't this like a downgrade? I took a look and saw that .npmrc contains only one line. My friend likes to do custom stuff with Meshtastic and GO; after that, he would need to build custom protobuf.

KomelT avatar Jan 23 '25 17:01 KomelT

In any way. The current version of protobuf on JSR is 2.5.9. The latest protobuf version in the repository is 2.5.19. It might be time to update so we can add new features to the web UI.

KomelT avatar Jan 24 '25 09:01 KomelT

Sorry, I'll explain the direction a little more clearly and the reasons behind it.

Previously, we were publishing directly to BSR (Buf Schema Registry), whilst it did work. One major issue we faces was that not all languages and runtimes played nicely with third party package registries. So instead of letting the codegen and publishing happen remotely, we could do it all within our CI/CD workflow. Currently, we provide language specific artifacts per build (currently C, rust, typescript) it's super easy to add more. Those however are secondary to the main effort which is to publish the generated code to first party package registries such as JSR and Cargo. If you have other platforms you want supported, please let us know and we can make it happen.

sachaw avatar Mar 24 '25 06:03 sachaw

I am using this buf registry so it would be extraordinarily helpful to me if this would be fixed.

wz2b avatar Apr 13 '25 23:04 wz2b

We can stop publishing to bsr as we aren't using any code generated from there anymore.

You aren't, but others (like me) are still using it. It looks like it's an easy fix, there's some secret that just needs to be re-configured in CI.

I'm going to add a few other comments in favor of keeping this:

  • It seems like very low effort
  • It lets me run the code generation any way I want ... for example I directly generate .ts objects, not .js + .d.ts as I really don't need the .js part
  • In order to use "@meshtastic/protobufs": "npm:@jsr/meshtastic__protobufs@^2.6.2" as a dependency I need to make an adjustment to tsconfig.json to make them aware that it's a module
  • What's released there is stuck on 2.6.2 which is still a few tags behind.

I would rather you keep publishing to the buf registry in parallel if possible.

wz2b avatar Apr 13 '25 23:04 wz2b

We can stop publishing to bsr as we aren't using any code generated from there anymore.

Regardless of that I think it would still serve as a very good source of documentation since it's nicer to use than looking at the files directly. Also it's a bit misleading that it's linked in the README without any warning that it contains outdated documentation 🤔

r41d avatar Apr 25 '25 21:04 r41d

This issue confused me as well. I was about to start developing new features for the Web Client, but I couldn’t find the latest Protobuf TypeScript build—until I discovered the #663

chihosin avatar Jun 08 '25 04:06 chihosin