port to a different language, e.g. Rust?

[yu-re-ka]

I saw there is a rust branch, and I would be interested in contributing to that effort, if it's still planned

[derhuerst]

Huh, good question!

I have started porting hafas-client to Reason, but didn't push through. Also, although I'm not as convinced by the selling points of TypeScript as many others, Deno seems to be a step in the right direction from the current state of the Node.js ecosystem.

I don't want this to evolve into a language/stack flame war though, like so many other porting discussions on the Internet. Rather, I'd like to pick a tool that cleanly supports these properties while also holding me/devs back rarely:

• keep hafas-client highly extendable/customisable – Several profiles modify the default behaviour in different ways:
  • opt into (or out of) features via flags (e.g. the DB profile)
  • change low-level HTTP/network behaviour (e.g. the SNCB/NMBS profile)
  • override select parsing functions with custom one (e.g. the BVG profile or the DB profile)
  • pass endpoint-specific parameters (e.g. the DB profile)
  • via a combination of the above, change hafas-client's behavior entirely (e.g. the WIP rest.exe support)
  • wrap hafas-client's API (e.g. cached-hafas-client & observe-hafas-client)
• JS/browser interoperability – Several Node.js-based libraries and tools use hafas-client, and they should continue to do so with a reasonable amount of friction.
  • a Promise-based API similar to the current one
  • ability to work in the browser using the fetch API
  • runtime introspection of a hafas-client instance should be possible (not just ahead-of-time via e.g. Rust traits), as done by hafas-rest-api
  • a machine-readable description of each functions parameters & options, to be used e.g. by hafas-rest-api's list of query parameters
• formalise the FPTF part of hafas-client's API, e.g. using types
• prevent runtime bugs – Although most of the bugs in hafas-client are due to previously unknown HAFAS response formats, some of the bugs could have been prevented by static type checking.

When evaluating several staticly typed languages, I quickly noticed some of their trade-offs though:

• Rust requires a bit of tooling publish to npm with both native code and WASM.
• TypeScript has a complexity toll on most tooling in a project (e.g. testing, linting, DX, publishing).
• Reason is not well-known, so the chance of long-term maintenance is lower.

---

Personally, I've been dabbling with Rust every now and then, but I don't have a lot of experience with it. I like its design principles, focus on quality and good interoperability with the JS/web ecosystem;

But for two reasons, I haven't moved forward porting hafas-client to Rust so far:

• My lack of experience in writing medium-sized Rust projects means that I don't have a "guiding gut feeling" for how to implement hafas-client's structure as idiomatic Rust code. I would certainly get things working, but I don't have an approach yet for how to keep all the properties listed above.
• Because I'm not that experienced, I'm slow; The process is much less natural than in Node.js, where I can express a domain concept without thinking much about the language.

[derhuerst]

Let me also state non-goals of this project:

• achieve great stability, no matter the cost
• follow the latest hype (e.g. TypeScript or Rust with no compelling reason to use them)
• use an elegant & well-suited by unpopular/esoteric language

[yu-re-ka]

I have made significant progress on a Rust port with hafas-client / FTPFv2-draft compatible output. You can see the code in the rust branch of my fork. It can do locations(), journeys() and refresh_journey() with DB and a newly added SNCF profile for now.

Would you be interested in adopting this code upstream? If not, I would probably split it out into a seperate repo.

[yu-re-ka]

My rust library is now living here: https://cyberchaos.dev/yuka/hafas-rs It is also used with javascript and webassembly in https://trainsear.ch / https://cyberchaos.dev/yuka/trainsearch. This topic is solved for me.

[derhuerst]

I have made significant progress on a Rust port with hafas-client / FTPFv2-draft compatible output.

🎉

I will try to read through it soon, and consider how much additional complexity will be necessary for a) new features like https://github.com/public-transport/hafas-client/pull/134 and b) all the subtle ways that individual endpoints are special.

Would you be interested in adopting this code upstream?

If I consider the code base to be maintainable by me (I don't have much Rust knowledge), definitely!

[Schmiddiii]

What is the status on this, is there still interest in a Rust-port? Looking at https://cyberchaos.dev/yuka/hafas-rs/, the code already looks pretty good, with one obvious pitfall, the project is in its current state without a license (and therefore not really usable anywhere). @yu-re-ka you will probably need to adapt the ISC-license from the JS-code. I also see some other minor issues like missing loyality-cards, but I think it could be easily extendable.

I am also really interested in a Rust-port for my application DieBahn, which until now used "transport.rest" but received some requests to support additional providers. I can help maintain it if needed.

[derhuerst]

What is the status on this, is there still interest in a Rust-port?

Yes, I'd be interested!

Looking at https://cyberchaos.dev/yuka/hafas-rs/, the code already looks pretty good, with one obvious pitfall, the project is in its current state without a license (and therefore not really usable anywhere). @yu-re-ka you will probably need to adapt the ISC-license from the JS-code.

Yeah or MIT. ISC is the same, except the MIT mention.

Before we can use hafas-rs as a fully equivalent drop-in replacement, there's more to be done though:

• porting the tests,
• porting all profiles,
• porting the docs.

Nevertheless, thank you for the great work so far @yu-re-ka! 🧡

I'm not sure I can contribute much to this effort, given my lack of time and knowledge of Rust, but I'd look forward to eventually switching to the Rust implementation of hafas-client once it is on par feature-wise!

[yu-re-ka]

with one obvious pitfall, the project is in its current state without a license

The Cargo.toml specifies the license, it is AGPL-3.0, though I am currently thinking about changing it to the EUPL. I am not planning to provide my implementation that I worked on in my free time in a non-copyleft license. I should definitely add a mention of the original hafas-client in the readme though, where I will also mention the more permissive license of that implementation.

[yu-re-ka]

I have since switched back to the JS hafas-client for trainsear.ch. Ironically I started hafas-rs because I thought it would be better suited for the server backend of trainsear.ch, but in the end I abolished the stateful backend and replaced it with a simple mgate.exe proxy and put all the parsing into the client. I could do this since I figured out a way to kind of efficiently encode the journey refreshTokens into a base64 url: AAFC6nAAAAH1G3oAF-ymATQDBEQzMDEBAAFwkXsAYO-mAXsAB0lDRTEwMDMB -> ¶HKI¶T$A=1@L=7400002@$A=1@L=8002549@$202209121551$202209130531$D301$$1$$$§T$A=1@L=8002549@$A=1@L=8098160@$202209130552$202209130755$ICE1003$$1$$$

And now in the javascript client code, hafas-client is better suited than a Rust solution because of the expense of copying around data between the JS and the WASM world. When polyline parsing is enabled, the resulting object is very large. I had to make some changes for hafas-client to work in the browser, mostly to the fetching code and removing some nodejs specific functions. I think the hafas-rs library is still interesting for server use cases though.

[Schmiddiii]

The Cargo.toml specifies the license, it is AGPL-3.0

Did not notice that. Normally, a license is specified in a LICENSE-file in the root of the repository.

though I am currently thinking about changing it to the EUPL.

That would be good, as with the current AGPL I would have to re-license my project from GPL to AGPL, with EUPL (as far as I understand), I can keep the GPL.

[derhuerst]

And now in the javascript client code, hafas-client is better suited than a Rust solution because of the expense of copying around data between the JS and the WASM world.

Fair enough. I'd argue though that having high-quality & maintained alternative clients is a great benefit. Of course this only applies if you (or someone else) feel like this is worth your time!

I had to make some changes for hafas-client to work in the browser, mostly to the fetching code and removing some nodejs specific functions.

Interesting, because last time I checked if hafas-client works in the browser (a few years ago by now), tools like browserify and webpack automatically shimmed all Node.js-specific APIs (Buffer, net.isIP, crypto.randomBytes), and the isomorphic packages such as create-hash, pinkie-promise, fetch-ponyfill work in the browser anyways.

There are some aspects that I have added recently though (proxying, localAddress) that may not work in the browser. Would you mind opening a new Issue to make them browser-compatible again?

[Schmiddiii]

Today I played around with hafas-rs a little more and it is a pretty good fit for my application. There were just some minor adjustments needed, e.g. deriving some more things, implementing bike-accessible, loyalty cards, product names for lines but nothing too major. Nevertheless, before I am willing to publish any of my changes, I would prefer a license-change as @yu-re-ka suggested to EUPL, as AGPL would not fit with my application. Then I am willing to upload my changes to GitLab and try to bridge the gap between hafas-client and hafas-rs.

[yu-re-ka]

Disclaimer: I am not a lawyer, and this is not legal advice.

Both AGPL and EUPL licenses have "network distribution" / "SaaS" clauses requiring you to provide the source code even if hafas-rs is only running as a service over the network. If using a EUPL licensed component within your application, the "network distribution" / "SaaS" terms will stay valid for the part of the application that was EUPL licensed, so you would have to provide the source code of hafas-rs only when your application provides a service over the network.

Source: https://en.wikipedia.org/wiki/European_Union_Public_Licence#Is_the_EUPL_%22Strong_Copyleft%22?

Since none of the compatible license prohibits the strong reciprocity implemented by the EUPL (obligation to publish and share the source code of derivatives, even distributed through a network) the copyleft resulting from the EUPL can be considered as strong.

After looking into this more closely, I will actually dual-license hafas-rs: You can choose to license it either under EUPL or AGPL.

[Schmiddiii]

Thanks again for considering the relicense. As my application will not be networked (talk to your library over the network, I directly call it from my application), AGPL will require me to re-license. But for EUPL, this does not seem to be the case (Source, but also not a lawyer here):

Compatible means that the work covered by the EUPL can be used/merged and distributed in another work covered by GPL-2.0, GPL-3.0, LGPL, AGPL, CeCILL, OSL, EPL, MPL and other licences listed as outbound compatible in the EUPL Appendix ...

I am still waiting for a commit that relicenses the repository at https://cyberchaos.dev/yuka/hafas-rs/ though.

[yu-re-ka]

Apparently the EUPL is like LGPL in the regard that it does not have virality (which lawyers are arguing if these terms of GPL and AGPL are even valid in European jurisdiction). And like the AGPL in the regard that it has the SaaS terms.

Your source also notes this:

Compatibility does not reduce the licensor obligations applied on derivatives, like the publication of the source code or the coverage of SaaS

So be sure to include a link to the source code of the hafas-rs library if you use it, no matter if on the client or server side, and if you modified it link the version of the source code as it is used in your application.

I pushed the dual-license change, so have fun ;)

[yu-re-ka]

Oh, and also feel free to submit patches to hafas-rs to me by email ([email protected]) or request an account on my GitLab.

[Schmiddiii]

Thanks. My forked version can now be found here. Feel free to just take commits I made.

[Schmiddiii]

Just a small status update. I have now gone through all profiles in hafas-client and autogenerated most profiles for Rust. The current results (from very basic testing):

Total hafas-client: 44 Working: 18 Version error: 5 Auth error: 6 (Probably MicMac) Always "no connection": 5 Custom Certificate: 3 Skipped: 1 Error H890[_R]: 2 Does not reply to some request: 3 (Must have missed while converting): 1

The version errors and auth errors are probably the biggest problem. For version errors I have no idea how that happened, auth errors are probably just missing the MicMac implementation in Rust. The "no connection" might be because either the profile does not support getting connections or because I have no idea what connections exist for those profiles and always just autocomplete. The custom certificates will be a little bit more problematic in the future. But I am sure I will figure out a solution. I have skipped one profile (BVG) as it had too much custom code for now. I got one profile with Error H890 and one with H890_R (no idea what that means) and three profiles do not even reply to my requests (also no idea why). Progress can be seen here with the currently working/not-working "list" here and things that still needed to be done inside the code for the profiles.

Sadly I probably will not have time to look into many of those issues in the next few months, but I definitly intend to continue improving this afterwards.

[derhuerst]

I have now gone through all profiles in hafas-client and autogenerated most profiles for Rust.

FYI: There is the transport-apis repo, which hafas-client pulls its profiles' base data from. You could use that one too for hafas-rs. It does not specify the custom logic needed to expose a specific endpoint's special features, of course.

Version error: 5 Auth error: 6 (Probably MicMac) Always "no connection": 5 Skipped: 1

Would you mind opening a separate Issue about this?

Error H890[_R]: 2

H890 is the error code for "journeys search unsuccessful".

https://github.com/public-transport/hafas-client/blob/0dc230837a56d1b013234e2db252bfe06a178a12/lib/errors.js#L92-L97

[derhuerst]

With hafas-rs, there is a Rust port. Thanks @yu-re-ka!

There are also other ports/clones like fshafas (F#), pyhafas (Python), hafas-client-php (PHP), as well as TripKit (Swift, also covers other API flavors), kpublictransport (C++, also covers other API flavors) and the venerable public-transport-enabler (Java, also covers other API flavors). bahn.expert & iRail also have HAFAS clients.

I don't plan to port hafas-client for now, it doesn't seem like the most effective use of time at the moment, given that there are many other problems to tackle in the public transport domain; That may change though. Therefore, I'm going to close this Issue.