Raku
Rakudo releases (and probably zef too) need a long-term support strategy
As more and more Raku applications enter into production, those applications are probably version-locked, so that they're guaranteed to work in a specific version of the language. However, we don't offer support for specific versions, not even Rakudo-Star ones. If some bug or critical security update is created, it's only applied to the next release, not backported to whatever release might be the one used by that (hypothetic) Raku application in production.
The same might be applied to zef, too. New versions are not guaranteed to work in old Rakudo releases, and old versions of zef are not guaranteed to work with the new repositories (which were changed if I remember correctly after the server meltdown in 2018? 2019?). So there's a catch-22 situation here. You might have a zef version working correctly, only to find out it can't download new versions of modules, even if they could work.
Long story short: We should try and designate some version along the year a LTS version, and support it for 5 years or whatever is thought to be reasonable. At the same time, designate some zef version that is guaranteed to work with it also LTS and try and support it for the duration. Maybe also give them some specific designation, 6.d.1 or whatever so that they stand out among the rest.
I don't think this is a viable solution due to the dynamic nature of the ecosystem.
I think the way forward is packaging an application and its dependencies together, maybe even with a Raku runtime. I played a little with the idea now that Rakudo is relocatable, but the C dependencies where hard.
Every ecosystem evolves, but you still know which versions are the ones that work with your application. Imagine however that you have your application working with Foo 2.3.1 and Rakudo 2019.10. A bug in Rakudo 2019.09 is discovered, that affects every Rakudo up to the current one. The but is fixed in 2021.01, which includes a series of improvements, some of which are incompatible with Foo 2.3.1. You need to either path Rakudo, or patch Foo so that it works with the new version, of both. Any way, you're in for a bit of yak shaving. Even if packaging everything together is a solution that helps reproducibility, you still need a long-term support strategy for this kind of scenario.
I am looking for a quick win to get you the magical 80% there while keeping the constraints in mind regarding the young age of the ecosystem and the relative small size of the community. I'm looking at things like Perl's App::Fatpacker and Carton or Golang's new module system based around minimal version selection (that took them a very long time to get it right).
I think that adding a LTS Rakudo to the mix would require a sizeable time investment in devs, fixing bugs in 2 branches and releasing it. At the speed Rakudo advances it would take long to make this process more difficult with each release. It would be great, sure, but I am not sure if it's viable. Maybe a good discussion candidate for the RSC. Add to the discussion the new role of Rakudo monthly: would be have enough users to test drive the new features/fixes?
When talking about modules, it's even more difficult. We can add LTS by example to the travis testing mix, but still there are no warranties people would care about 5 year old releases (for the sake of the argument). And with this said, there will be many cases where the user will have no control which Rakudo version they'll run and stick to whatever the distribution or packagers like me supply.
Any way, big thumbs up for thinking about this!
Quite clearly this would need resources. How many, however, will depend on the bug-fixing strategy. Probably the best would be to prepare patches for the bugs based on the first release where they appeared, and then apply them to every LTS release since. Backporting bugfixes might be totally more difficult. On the other hand, LTS releases will need to be branches, instead of tags, which is all we have now. I'm not saying we should have many releases. Normally, LTS releases are done once every two year or so. Still, I really think we should nominate some old release (like the first 6.d, for instance) LTS and support it for a few more years.
Who would do the work for LTS releases? Who would pay for this work?
So far our strategy has simply been: do not break existing code. Upgrading rakudo to a newer version should always work, so sticking to a specific release should not be necessary.
I'm leaning on the side of "we aren't big enough for LTS yet". I suspect there are other less work options that will also help users manage Rakudo bugs for their use cases. Promoting Containers being one of them. Finding ways to switch Rakudo versions and keeping the installed modules might be another. Maybe we can come up with more?
Containers are precisely what is bringing me here. I can't seem to be able to install zef in the container environment created in 2017.05. Quite clearly if I had built that container back then it should still be available and working, but that would have implied to create a long-term strategy in the first place. It's a Catch-22 situation here. The same would be my answer to @niner: we can't have the resources to have wide support in the industry unless we devote some resources to strategies that help Raku win wide support in industry.
Upgrading to a newer version should always work
Except it does not. Again, during the creation of the set of containers, I've bumped into problems with Getopt::Long, which simply stopped working somewhere along the 2017::* releases. Before getting to that, I had to downgrade from the latest release 0.3.4 to 0.2.0 in the 2018.*. Every release introduces new features. It will simply stop working with older releases. It would be a best practice to tag all your releases, and say also which releases are compatible. Which is possible-ish right now, since there are test containers for many Rakudo releases. But not before, so it's simply not done. Baseline is you can't say your module always work, even (relatively) critical ones like this one, unless you're able to test at least the LTS releases.
And obviously, as stated in the OP, it's going to be extremely difficult for any company to start using in production unless they have the resources to maintain all the used modules when they start breaking with future releases of Rakudo. So the situation is really Catch-22. We might never get big enough to need a LTS unless we have a LTS strategy.
Except it does not. Again, during the creation of the set of containers, I've bumped into problems with Getopt::Long, which simply stopped working somewhere along the 2017::* releases. Before getting to that, I had to downgrade from the latest release 0.3.4 to 0.2.0 in the 2018.*. Every release introduces new features. It will simply stop working with older releases.
Wait, that's two different things. I said, installing a newer Rakudo must not break your existing code (Rakudo backwards compatibility). You now give examples of new module versions not running on older Rakudo releases, that's something else (Rakudo forwards compatibility or module backwards compatibility).
It would be a best practice to tag all your releases, and say also which releases are compatible. Which is possible-ish right now, since there are test containers for many Raku versions. But not before, so it's simply not done. Baseline is you can't say your module always work, even (relatively) critical ones like this one, unless you're able to test at least the LTS releases.
If the module version you'd like to use doesn't work on older Rakudo, then simply upgrade Rakudo to a version where it runs. Since according to our policy, upgrading Rakudo must not break your code, you should be fine.
Containers are precisely what is bringing me here. I can't seem to be able to install zef in the container environment created in 2017.05.
Containers are the top case for me where you want to pin the runtime and the libraries, what other programming languages call "vendoring". We don't have a standard way to do that. This would fix your problem on a reproducible way, while just updating Rakudo would fix your new incompatible modules problem.
(This does not mean that LTS wouldn't be useful, but that the example is not a good illustration.)
WRT modules breaking: I put high hopes in fez (the new ecosystem tony-o is working on). Similar to the CPAN eco, releases will never change and not disappear (in contrast to p6c). Version pinning will then be possible.
We're looking too much at compatibility problems here (me included). That's not the main, critical issue thouth. Security bugs are (or might be). We've had no major security bug so far, but when we do, what's the way forward? Fix it in the next issue, and have everyone upgrade? (we'll probably lose quite a few users in the process). Fix every release so that we don't? (That's going to need quite a few resources, even more than maintain one or two LTS releases). Even Perl has a (kinda) LTS strategy. So I think we should, even if we go for a minimal one: Support the first release after letter switch. @patrickbkr it will solve #72 (but you could actually "post" different versions to p6c if you so wanted), but still the same problem. What if there's a critical security issue with the Rakudo release they're using and you need to upgrade?
On Sonntag, 27. Dezember 2020 17:43:57 CET Juan Julián Merelo Guervós wrote:
We've had no major security bug so far, but when we do, what's the way forward? Fix it in the next issue, and have everyone upgrade?
Yes.
(we'll probably lose quite a few users in the process).
Why would we?
Fix every release so that we don't?
Unless someone demonstrates why it's necessary, we won't.
What if there's a critical security issue with the Raku version and you need to upgrade?
Then upgrade?
WRT modules breaking: I put high hopes in fez (the new ecosystem tony-o is working on). Similar to the CPAN eco, releases will never change and not disappear (in contrast to p6c). Version pinning will then be possible.
I don't want to derail the issue, so I'll inform on IRC. Does the CPAN-like ecosystem mean losing the single best feature of the Raku ecosyatem: easy publishing through github?
On Sonntag, 27. Dezember 2020 17:43:57 CET Juan Julián Merelo Guervós wrote: We've had no major security bug so far, but when we do, what's the way forward? Fix it in the next issue, and have everyone upgrade? Yes. (we'll probably lose quite a few users in the process). Why would we?
As said above, theoretically upgrading Rakudo won't break any module. In practice, it does. It's supposed to when you change major version, as it has happened from 6.c to 6.d. Just imagine someone relying on a module that broke in that transition. They would have to patch the module if they want to continue using the latest version of Rakudo. With a LTS strategy, we would have kept patching 6.c up to... I don't know, maybe next year. We're still in time, as we transition to 6.e, to give long term support to a specific release, so that people can rely on it working some years into the future, and so that they can continue using any module they use without fear of breakage when they upgrade (because they won't need to upgrade, at least for some time)
Fix every release so that we don't? Unless someone demonstrates why it's necessary, we won't. What if there's a critical security issue with the Raku version and you need to upgrade? Then upgrade?
Again, it might break some modules you rely on. There are 300 modules in the ecosystem (probably more if the tests are done now) that have stopped working, probably because they relied on some behavior that became obsolete with some release (6.c or 6.d) or maybe because they were using some ambiguous feature that was disambiguated (in a different direction) or some undocumented feature that simply changed. We can't of course support all undocumented features, or indeed any of them. But LTS support is about stability: you know that the release of Rakudo you rely on will continue working, at least for some time, while you have time to upgrade modules that go with it or develop your own, whatever. Really, I don't see this as a big issue. It's a (relatively small) investment, and it will show commitment on our part to users.
On Sonntag, 27. Dezember 2020 19:33:44 CET Juan Julián Merelo Guervós wrote:
As said above, theoretically upgrading Raku won't break any module. In practice, it does. It's supposed to when you change major version, as it has happened from 6.c to 6.d.
No it's not supposed to. But then, you are talking about Raku releases (of which there were exactly 2 so far: 6.c and 6.d) on the one hand, but raising examples about breakages caused by Rakudo (of which there were dozens of releases) on the other hand. That's two very different things.
Of course new Raku versions are not necessarily backwards compatible with previous versions and they don't have to. That's why you state explicitly which Raku version your source code expects by saying "use v6.c;" or "use v6.d;" at the start of your code. This is how we ensure that even breaking changes don't cause any harm and old code stays functional.
Just imagine someone relying on a module that broke in that transition. They would have to patch the module if they want to continue using the latest version of Raku.
No, they wouldn't. The module would still run just fine with the older language version.
With a LTS strategy, we would have kept patching 6.c up to... I don't know, maybe next year.
Actually we are patching 6.c and have been doing that since Christmas 2015. That's what the 6.c-errata branch in roast is. Same is true for 6.d of which there is a 6.d-errata branch that gets commits when we discover that that language version has a bug.
We're still in time, as we transition to 6.e, to give long term support to a specific release, so that people can rely on it working some years into the future, and so that they can continue using any module they use without fear of breakage when they upgrade (because they won't need to upgrade, at least for some time)
That's exactly what our current policies already should give you.
Again, it might break some modules you rely on. There are 300 modules in the ecosystem (probably more if the tests are done now) that have stopped working,
Wait, what?! I've been present (if not more actively involved) for every single Rakudo release since 2015. At no point has anyone stated that we broke hundreds of modules? We test every release against all ecosystem modules precisely to avoid such a scenario. Very, very few modules breakages were deemed ok, because those modules were already quite broken before the release and only bugs caused the breakage to stay hidden. Or they were dibbling into Rakudo's internals.
probably because they relied on some behavior that became obsolete with some release (6.c or 6.d) or maybe because they were using some ambiguous feature that was disambiguated (in a different direction) or some undocumented feature that simply changed. We can't of course support all undocumented features, or indeed any of them. But LTS support is about stability: you know that the version of Raku you rely on will continue working, at least for some time, while you have time to upgrade modules that go with it or develop your own, whatever. Really, I don't see this as a big issue. It's a (relatively small) investment, and it will show commitment on our part to users.
We already have large commitment to our users by ensuring that we do not break their code at all in new releases. How much more commitment can there be?
What are those 300 modules that are supposedly broken by Raku or Rakudo releases? How could they even be broken by a release of a language specification? Why didn't we catch the breakage when testing Rakudo releases?
So there's indeed long-term support for 6.c in the shape of that branch. So when you download 6.c, it's actually the updated branch? Same with 6.d in the future? So there's indeed a LTS strategy, only it's not stated in so many words. Until when will 6.c be supported that way? (Just in case it's not clear enough, that's exactly what I'm asking for)
WRT to the questions in the last paragraph, there's this issue #39 which mentions that precise number, although there seems to be 215 for the time being. Breakage can come from all kind of sources, but it's not fixed if it's abandoned. Also, testing extensively with all ecosystem modules is relatively recent, it didn't happen, as far as I can tell, a long time ago. Baseline is, modules will break (due to all kind of legitimate or non-legitimate reasons) and they do. The answer to the last question is, I guess, Rakudo releases started to be tested on all ecosystem modules at one particular point in time. Before that, all kind of things happened.
On Sonntag, 27. Dezember 2020 20:14:00 CET Juan Julián Merelo Guervós wrote:
So there's indeed long-term support for 6.c in the shape of that branch. So when you download 6.c, it's actually the updated branch? Same with 6.d in the future? So there's indeed a LTS strategy, only it's not stated in so many words. Until when will 6.c be supported that way? (Just in case it's not clear enough, that's exactly what I'm asking for)
Yes, it is. But why is that important? Even if we completely removed the whole roast repository, nothing at all would change for any user whatsoever. Roast is for compiler developers and maybe documentation authors. It may have value as a repository of example code. But in no way is it connected to any end user code written in Raku.
WRT to the questions in the last paragraph, there's this issue #39 which mentions that precise number, although there seems to be 215 for the time being.
This list doesn't mean that those modules were broken by a Raku or Rakudo release (still not sure which of the 2 you actually mean!). That's just a list of modules that could not be installed on a certain test system. The reasons could be plenty. Indeed there's not even a requirement for a module to have ever been installable to enter the list. Or take https://github.com/Raku/ ecosystem-unbitrot/issues/28 or https://github.com/Raku/ecosystem-unbitrot/issues/54 which show that a module doesn't even have to be broken in any way to make it into the list. The whole problem seems to be that the module depends on some native library that wasn't installed on the test system. Not broken at all and least of all by Rakudo.
Baseline is, modules will break (due to all kind of legitimate or non-legitimate reasons) and they do.
Please demonstrate this.
The answer to the last question is, I guess, Rakudo releases started to be tested on all ecosystem modules at one particular point in time. Before that, all kind of things happened.
Even if we take your assertion without requiring any facts to back it up, your explanation still only shows that we may have broken stuff years ago. That still doesn't show that we do so today. So why would we need any new effort today?
OK, let me get back to square one here. I probably didn't make my case a good one. So let's clarify stuff here and see if I can gather some support. Let me make clear that a LTS policy would be beneficial for the whole ecosystem, and would probably not suck many resources out of the usual release cycle.
First, a long term support policy basically means supporting (via bug fixes) certain releases of a product for a period of time that can be anything at all. LTS support periods can overlap, or they simply can go along with short-term support periods. The main tenet of a LTS policy is to show commitment, by the release community, to certain versions (and clients) so that they can rely on having a stable (let me emphasize this, stable) production platform. As the Wikipedia page indicates,
... It also alters the type and frequency of software updates (patches) to reduce the risk, expense, and disruption of software deployment, while promoting the dependability of the software.
New releases can be guaranteed to be compatible with old releases. You still have to test them thoroughly before getting them into production, and this will need resources from the company. LTS releases don't. Patch, deploy, it's not going to change at all. Of course some testing will be also needed, but since no new features are introduced, you'll get all the benefit of bug fixes without the uncertainty of new (or dropped) features.
Any language I know of, and certainly those that are listed on the Wikipedia, have one. Even things that are not so mission-critical as browsers have one; they are still application platforms and you probably need to rely on them. "Just upgrade" is not really acceptable for someone that has a whole application platform that depends on Raku and suddenly discovers a bug. Then all of a sudden they've got into Rakudo 2020.12, with a slew of new capabilities (and other possible bugs) they need to test all over again, extending their release cycle. And possibly breaking some modules that are used with the existing running version.
Because it can be said that, if modules are developed correctly and according to published API, they won't break. Well, I have news for you: they don't. I've had to patch modules all over the ecosystem when I upgrade to a new version of Documentable. Test don't have complete coverage, modules use undocumented features...
... And they also use features that are eventually deprecated. There are almost 300 commmits that use the word "deprecated". Features are deprecated from one release to the next, and rightly so. "undefine" was removed here from a specific place. That might have been used in some published code (although, TBH, I haven't found any)... or in some private code that will stop working.
The strategy of testing all modules (or at least the core ones) in every release is a good one. However, that does not account for private code, which might break in new releases and get a breakage for free when all they want is to avoid some bug that has been fixed in the last release.
So, again, this is my request. Starting now, create as many LTS releases as we can afford, and keep fixing bugs on them for a reasonable period of time. If we can create only one every four years, that's fair enough. But this would really give Raku reliability and make it more trusted in the minds of companies using them in production.
The first thing coming to mind when considering the LTS is a case where a bug fix for a problem existing in an old LTS version is based on an massive previous change. As the latest example I could point to the fix of #1285 which ended up as a major revamp of coercions. Now, to get it fixed for LTS would either require merging the whole coercion thing or developing a new solution. Or leaving it unfixed altogether which is perhaps the best solution of all. Ok, #1285 wasn't a critical one. But what if it was?
My point here is the same @niner made a couple of comments above: we simply don't have resources for LTS. Not until Rakudo gets some kind of sponsorship from a major market player.
Until then the best strategy is to:
-
clearly separate Raku and Rakudo version in mind
-
authors must stick their modules to the Raku version they guaranteed to work with
With regard to this part I think we're gonna have a round of submitting PRs to modules which rely on
use v6when we eventually transition Rakudo to 6.e default. -
keep erratas for older Raku versions. It's hard to say when 6.c will go out of support scope, but it depends on when supporting it would start taking too much time.
-
keep testing the ecosystem on a regular basis
That's the best we can do so far.
@vrurg again, this is clearly a Catch-22 situation. We're unlikely to get any major sponsorship unless we have a LTS strategy so that some company with enough resources will see this as a stable platform to put major systems into production.
The point about resources is fair, but again, I think it's worth the while to divest some resources from the actual release cycle to put it into a LTS cycle. This is also related to #203 and also #5 (which is closed, but maybe needs to be open). Maybe the release that's destined to be LTS should be a bug-quashing, PR-accepting one; since it's not introducing any new feature, it should be easier to backport error fixes to it. The resources needed to maintain this specific release could be taken by skipping one monthly release, for instance.
You're also saying that authors must stick their modules to the Rakudo release they're guaranteed to work with, but the problem is, which ones should they test? All and every one? With the old Travis configuration, it takes several minutes to test every one. If you use jjmerelo/raku-test, which is extended now down to 2017.06, there are more than 20 versions to test, consuming precious (free tier) CI budget. Having LTS distributions means having some solid post to pin your distribution to, and also that you need to test only for LTS distributions, and then the current and maybe the nightly (which you can also do now, BTW).
I'm not saying that regularly testing the ecosystem is not a good idea; it's an excellent one, and it will help avoid breakage. But the code I'm talking about will typically not be in public repositories.
So what I'm saying is: let's pick a monthly release, let's say .08 (so that it's the one in the anniversary of the creation of the language), let's skip .07 (so that people can prepare their talks for Perlcon without too much stress), let's make it a bugfix only release, and let's support it for 12 months, up to the next LTS release. And then let's say we make the first 6.e release a longer TS with support for 3.14 years.
Even if we don't do LTS and make it only a bugfix release, it's something I've been wanting for years, from the issue #5 I've mentioned.
@JJ please, can you finally make clear if you are talking about Raku or Rakudo?
There are so far precicely 2 Raku versions. 6.c and 6.d. A module author only ever has to test exactly one Raku version. The one that the module was developed for. That's it. You declare the langauge version, run your tests, be done. That's certainly not too much to ask.
There is no Raku version 2017.06.
@JJ please, can you finally make clear if you are talking about Raku or Rakudo?
There are so far precicely 2 Raku versions. 6.c and 6.d. A module author only ever has to test exactly one Raku version. The one that the module was developed for. That's it. You declare the langauge version, run your tests, be done. That's certainly not too much to ask.
You test with which release of what version? Nightly? The first one? Current monthly one?
Since the 6.c version is improved well into the 6.d releases, a good solution for this would be to provide rakudo-level switch that would "switch-off" 6.d features and make the current compiler 6.c only. You can do that now at the file level, but this means that testing for this would involve adding use v6.c to every single file (since it's a file-level pragma), up to and including the modules that are being loaded. A rakudo-level command line switch would effectively turn the compiler into a "6.c-only" one, and would allow testing for that version.
Since the 6.c version is improved well into the 6.d releases, a good solution for this would be to provide rakudo-level switch that would "switch-off" 6.d features and make the current compiler 6.c only.
This would be a solution for what? What would be solved by having such a switch?
You can do that now at the file level, but this means that testing for this would involve adding
use v6.cto every single file (since it's a file-level pragma), up to and including the modules that are being loaded. A rakudo-level command line switch would effectively turn the compiler into a "6.c-only" one, and would allow testing for that version.
What would be the point of forcing Rakudo to run a module that's written for a 6.d compiler with a 6.c compiler?
That would be as if we added a --python command line switch so one can conveniently run Raku modules using a Python compiler. Why would one want to do this? What would you expect to happen? Why do you think this would be useful?
Please, the language version thing is extremely simple: you choose a language version to develop against, add the corresponding use v6.whatever; statement to the file, and develop. No further action needed. Just make sure that you always explicitly state the required language version.
If you have a dependency that requires a certain language version, then that's it. Your users will need to make sure that their compiler and runtime support this version. There's no point in testing with anything older than that version.
Since the 6.c version is improved well into the 6.d releases, a good solution for this would be to provide rakudo-level switch that would "switch-off" 6.d features and make the current compiler 6.c only.
You actually have the switch. Not for CLI though, but use v6.c is what you ask for. Try finding any of 6.d or 6.e symbols in your namespace with the statement – you'll fail. Also, the internal code paths effectively cut-off any 6.d or 6.e specific behaviors in both the metamodel and the compiler code too.
Talking about the corporate non-public code, LTS doesn't guarantee it wouldn't be broken. In particular, a bug fix could break the code which somehow relied on the bug to work. The only way to avoid a situation like this is for the company to fill in an issue and make sure tests are included into the roast. With this in mind all they need is to be always specific about Raku version their code use. I.e. no way they use anything but use v6.c if that's what they expect from the compiler.
Maybe I'm mistaken somehow, but overall the versioning support in Raku makes it much less dependent on any kind of LTS than any other language around. I would even be brave enough as to state that language versioning support is the LTS we need.
Quite related to some parts of https://github.com/Raku/problem-solving/issues/365 actually. The moral of the story seems to be that most people (basically everybody apart from the about 5 involved individuals who just know how it's done by heart) have very different understandings about the versioning of Raku (not sure how much Rakudo is involved in that topic; software should probably depend on the language specification rather than Rakudo, anyway).
For Rakudo itself: the idea seems to be more or less continuous development, and ever-improving behavior according to the language specification. In this sense, it wouldn't matter what Rakudo version somebody is using - if a bug slips in, the choice is free to upgrade or downgrade to a different compiler version that implements the same language version, not containing the bug.
