Should rhombus be a superset of pyret?

[spdegabrielle]

Rhombus ⊃ Pyret ?

Bear with me

• A racket implementation already exists (admittedly it is a little behind Pyret on the webcloud)
• gives a whole bunch of students a powerful 'what next?' option without them going to Python.
• Pyret as a teaching lang (as a subset of Rhombus)

What are the problems with this idea?

[sorawee]

One major problem is that Pyret syntax was not designed to support macros, and even if we managed to find a syntax for macros, the existing grammar is likely going to strictly constrain the macro syntax that you can't use it in the same way you can in Racket.

When I was at Brown, we discussed about adding macros to Pyret for countless times. Pretty much all proposals failed. I don't know if anything has changed since then. @blerner would know more about it.

[blerner]

While I'd love to see Pyret be a strict subset of Rhombus, I should give some cautionary advice here. The Racket implementation of Pyret is incredibly defunct and is nowhere near compatible with current Pyret; for all practical purposes, I would ignore that implementation and start from scratch.

As for macros, yeah, this has been discussed a bunch of times, and we've never come to a satisfactory conclusion. There are technical and pedagogic challenges here:

• Pedagogically, Pyret's syntax-error messages are tightly coupled to its grammar; there is no reader layer followed by an expander, so the AST we parse comes directly from the parser, and we have some bogus productions in the grammar as hacks to enable better error messages.
• Technically, it's just really hard to have a good notion of macros while also supporting infix operators, postfix function application syntax, expressions that don't have to be delimited with parentheses, etc. Because there's no reader and expander, there's no notion of an s-expression corresponding to our unexpanded syntax. We've discussed having some kind of delimiter {macro-name: ...} to say "within this pair of braces, lex the tokens between them and feed them to macro-name to expand as it wants. This is vaguely akin to Rust's macro syntax, which would look like macro-name!{...}, more or less. We never discussed what underlying tech would be used to implement macros; we we stymied simply by how best to integrate it grammatically into our existing language.

But if you can figure it out, we'll happily borrow the solution! :)

[shriram]

To be clear, @blerner and I have somewhat different takes on what macros for Pyret might even look like.

Ben would prefer that they be constrained to a particular syntactic ghetto (as noted above). I wonder if it might not be possible to define several kinds of "s-expressions": just as Racket s-expressions are (mainly) "things between ( and )", maybe for Pyret we can define "things between ( and ), things between : and end, etc.", so there are several kinds, and you can define expanders over them.

It would be pretty complicated, and Ben has in the past raised objections for why this can't be done. (Plus there's always the mess of infix binary operators, which probably need to be handled specially…)

It's worth noting in this conversation that Ben is the expert on parsing, not me (-:.

Nevertheless, I conjecture that we could re-build Pyret syntax (maybe not the exact same language but something quite similar) starting from multiple kinds of s-expressions (rather than figuring out how to retrofit them), with some special handling of infix binops (which will complicate everything). Sadly, the infix binops are the non-negotiable part.

[jackfirth]

Several of these challenges seem like things that Honu's approach to macros in traditional syntax would be well suited for.

[spdegabrielle]

There seems to be a lot of alignment in priorities

• infix binary operators - I’m assuming this includes arithmetic, logic and concatenation of lists, strings, regex & other data types where binary operator for concatenation makes sense
• macros (honu mechanism as a path forwards)

Maybe a strict superset is too much - the teaching languages are not strict subsets of racket iirc.

Maybe a Dread-Pyret-Racket language that is distinct from Pyret (has macros), but easily transitioned to?

[Metaxal]

Reading all this strengthens my view and my preference that Rhombus should be "s-exp first, other syntaxes next". This doesn't prevent it from having a default supported alternative syntax that could be used as bait of course.

[shriram]

Pyret isn't just a syntax; it's an ecosystem. We have a large number of users, materials, etc. written around it.

There is nothing special about Pyret's syntax. In fact, it was explicitly designed to be as unspecial as possible. People generally like the syntax, and that's nice to know. But any number of other infix syntaxes could take its place.

The main value in Pyret is the ecosystem. That value cuts both ways. By having Pyret run both in its current form and as a #lang, the current community of students and teachers who use Pyret would get a second, amazing, implementation to run on (that works well for off-line use, is probably faster, etc.). In turn, the Racket community would gain many more users.

However, that only holds if the two languages are the same. People need to be able to drop code in from, say, Bootstrap materials, and have them work in either place. Otherwise, it would be dangerous to even call the result a "Pyret" because it would confuse people who don't know better, expect things to work, find that they don't, and get frustrated. Since many of those people are not very technical, their frustration would be especially problematic.

Of course, if someone wants to make a differently-named sibling language that begins by copying Pyret's pleasant syntax, they are more than welcome to. That wouldn't affect/impact the Pyret team in any way. Give us credit, have fun.

Personally, as someone who sits in both camps (indeed, everyone on the Pyret team has far more than a passing affection forr Racket), I'd rather see the same language in both places. Ways to adapt Pyret along Honu lines to enable macros, etc., would all be welcome. (Though, Pyret's current IDE is not going to add Racket support, so Racket code written inside Pyret won't ever work there. But that's not a kind of non-sameness that worries me. The person who takes existing Pyret code and starts writing Racket macros in/over it probably knows what they're doing. I'm only concerned about the needs of the person who sticks to pure, macro/Racket-inline/…-free Pyret.)

[spdegabrielle]

Ok

For Rhombus to be considered a superset of Pyret the following criteria are hard requirements;

• infix binary operators - I’m assuming this includes arithmetic, logic and concatenation of lists, strings, regex & other data types where binary operator for concatenation makes sense
• macros (honu mechanism as a path forwards)
• strict superset is essential, including replicating ecosystem functionality, to pyreteers can drop code in and transition with no changes

I’m assuming the following in my understanding of Rhombus;

1. Rhombus will be implemented in Racket
2. Rhombus will be a default alternative syntax.
3. Rhombus will a friendly syntax version of Racket (i.e. incl item 4)
4. Rhombus intended to have the full power of racket (macros, gradual types, full interoperability with racket, native code compilation, libs and toolchain)
5. Capability to freely mix racket and rhombus modules.

[bijang]

@shriram it would be amazing, in 3-5 yrs once rhombus comes to fruition, racketscript will (hopefully) be finished and PLAI 3 is finished, to consider re-unifying/re-aligning(with whatever htdp3 ,if any, turns out to be) the languages and education program with racket/rhombus, i'm very excited from the education front! wrt to rhombus as superset of pyret, in full knowledge that the current syntax may not be the final product, and in full disclosure that i havn't used pyret in an educational form, would it be necessary to use pyret as a subset at all? or use rhombus as a pyret equivalent to teach the same lessons now the syntax is more python-esque?

[shriram]

At this point Pyret's syntax is pretty set, though it is conceivable that if there are small changes we can make that would enable, say, clean macro systems, while still keeping the language "comfortable", we would definitely consider it. It's important to us to not constrain the beginner programming experience too much for the sake of a feature that we can appreciate but they cannot (e.g., making the language macro-friendly).

As for using Rhombus as a "Pyret equivalent", that would probably have to be done by someone else. We have a large amount of materials built around Pyret, and many teachers trained in it, and more of it being produced all the time (so way, way more will have been built over the next 3–5 years). Moving all that over is highly non-trivial.

Furthermore, Pyret is not just a "syntax". It has, for instance, wholly integrated features like tables and reactors (which a basic RacketScript will not be able to support, but one enhanced with Stopify could). In addition, there are now people integrating Pyret into other tools (e.g., some math education tool frameworks); all that work would also have to be repeated with Rhombus/web.

So not only is "switching syntax" hard enough, it's not just a matter of doing that. A lot of other things would also have to be built.

TBH, I have not been following the development of Rhombus. Therefore, I don't know whether compatibility with Pyret is a design consideration at all, or how much it has deviated…