initial candidate experimental prototype Rhombus

[mflatt]

Rendered

Here's an initial candidate experimental prototype Rhombus using shrubbery notation.

[jeapostrophe]

As you know, it's very big and it seems pointless to comment on little things.

I have lots of little comments, like want things like val, fun, and def because typing out function hurts. mut/mutable. expression_macro seems even more brutal. mac def or mac exp or mac bind would be pleasant. Similar forward is a cool feature but feels like a really painful name to write out constantly... makes me want to not use it at all. This is basically like define*, but presumably we can't do def* because it's using a weird Schemism and operators?

The propagation-of-dot-information discussion feels ad-hoc to me. I think it would be nice to be able to explain how the current prototype propagates as far as it does and how it would be easy (or not) to get that pushed further, rather than just say "Maybe it should go further". In other words, I feel like there should be a "dot protocol" that is discussed.

I'd like to hear how the syntax discussion does or does not comport with syntax classes and whether that is "nice" (or rather how hard a rhombus-wrapper is to make)

[sorawee]

s/forward/let/ :)

Pasting an installation guide here, in case anyone wants to try it out:

You need Racket >=8.2.0.5, so either install Racket from source or use https://snapshot.racket-lang.org. Any version of Racket should now work, after the latest update You need a pretty recent version of Racket, so either install Racket from source or use https://snapshot.racket-lang.org.

To get @mflatt's repo, run:

git clone https://github.com/mflatt/shrubbery-rhombus-0.git && cd shrubbery-rhombus-0
raco pkg install enforest/ shrubbery/ rhombus/

To get this PR (which as I understand is more stable, but will be behind @mflatt's repo), run:

git clone https://github.com/racket/rhombus-brainstorming.git && cd rhombus-brainstorming
git fetch origin pull/163/head:prototype
git checkout prototype
cd rhombus && raco pkg install enforest/ shrubbery/ rhombus/

If you have an out-of-date Rhombus, update to the latest version with

For @mflatt's repo:

git pull
raco setup --pkgs rhombus

For this PR:

git pull origin pull/163/head
raco setup --pkgs rhombus

[sorawee]

• Typo: a initial character

• The multiline comments are not greedy and can be nested (like OCaml, and unlike C). Not sure if it's intentional, but this is a cool feature.

• continue in the match tradition of requiring an escape to create a pattern variable

  You mean the quasiquote pattern in match, right? Cause otherwise, match doesn't require an escape, IIUC.

• Can a binding macro binds multiple variables at the same time?

[mflatt]

@jeapostrophe

I think def, val, fun, and let would work well. I'm not a fan of mut or mac. Typing 7 characters is already a pretty light penance for declaring a mutable variable. I agree that expression_macro and binding_macro are very long, and I'm not sure of the right direction. (Note that you can already use just define for expression_macro.)

I agree about the propagation-of-dot question. The parts here are some of the last pieces I put in place for the draft proposal, and I figured it was better to start a discussion than try to sort out more.

I expect syntax quotes to work well with syntax parse and syntax classes, and they work well for implementing all the predefined forms using Racket. The S-expression encoding of shrubbery notation is working the way it's supposed to.

@sorawee

Using let for forward seems nice.

Yes, I mean that ? is like quasiquote in match.

Yes, a binding form can bind multiple variables, as in Posn(x, y) used as a binding. It can also bind syntax, which is how p :: Posn as a binding communicates the contract Posn to uses of p.

I'll get corrections and clarifications in the next update.

[mflatt]

A thought on names like expression_macro: I'm so used to Racket that to distinguish things, I automatically resort to making long names with dashes or hyphens. But constructing hierarchical names with . may be a better way to go: expr.macro and bind.macro, or something like that. Using more hierarchy and shorter names fits well with #159, too.

[sorawee]

Yes, a binding form can bind multiple variables, as in Posn(x, y) used as a binding. It can also bind syntax, which is how p :: Posn as a binding communicates the contract Posn to uses of p.

I somehow missed the <&> example. That's exactly what I was looking for.

[sorawee]

binding_operator ?(¿a <&> ¿b):
  match unpack_binding(a)
   | ?(¿a_id, ¿a_matcher, ¿a_binder, ¿a_data):
       pack_binding(?(¿a_id,
                      build_anding_match,
                      build_anding_bind,
                      (¿a, ¿b)))

build_anding_match and build_anding_bind make sense to me, but I don't understand why a and b are treated non-symmetrically in the main binding_operator.

[mflatt]

The <&> arbitrarily picks the name suggested by the a pattern for naming a right-hand side.

For example, suppose you write this, where object_name is connected to Racket's object-name:

val f <&> g: function (x) x
object_name(g)

The <&> implementation makes the result f, even though g or some other name would be equally valid.

[mflatt]

The new draft uses def, val, fun, and let, and it changes require and provide to import and export.

Imports now involve a prefix by default to discourage “namespace dumping” (see #159). Supporting hierarchical names for imports and other purposes, such as expr.macro, requires an additional concept at the level of enforestation; it doesn't work well to view those dots as the infix . operator, but it does seem to work to add a notion of hierarchical names to the next layer down.

The idea of dot providers for the infix . operator is much more worked out in the prototype implementation. The proposal also explains more, but it's tedious, so that's pushed out to a separate linked document.

It's not clear that overloading . for hierarchical naming (like imports) and dot providers (such as field access) is a good idea. But it's intuitive if you don't look too closely, and it avoids using up another operator.

[jeapostrophe]

The required prefix is pretty painful for operators as you note. I can't really know without trying to program in it, but I feel like I might want the ability to only import some things without a prefix by explicitly naming them:

import:
  "posn.rhm":
    <>
    origin

posn.distance ( origin <> posn.Posn(1, 2) )

I don't understand why the dot-provider doc repeatedly uses the phrase "(with|has|) a structure-type contract"... why is it not "a dot-provider"? For example, why does A in struct Posn (x :: A) have to be "structure-type contract" rather than a "foo" where a "foo" may have an "enforcement" part and may have a "dot-provider" part, where "contract" is an easy way to get an "enforcement" part and "structure-type" is an easy way to get a "dot-provider" part? In particular, I want .first and .rest; I want classes to provide dots, but they aren't structs; I want to have a dot-providers that don't correspond to structures at all, but have layers of dot-providers.

In the dot provider doc, you talk about how a parenthesized expression defeats dot propagation. That's very unsatisfying and I feel like a syntax-property system could propagate the information out.

Finally, I don't understand why the . in imports "has to be" different from the . for dot providers. The import could be one binding and something like (#%app (#%dot p m) . args) could be a macro application by adding special cases to the expander. I'm not saying that this is elegant or clean or whatever. But I'm not convinced that "has to be" is the right phrase, I think it is "I like this because it makes A, B, and C simpler".

Typos:

• "tat"
• "Just like fun in JavaScript" should be function
• "exmaple" in dot provider doc

[mflatt]

The import and export forms need a lot of work, stil.

Yes, the dot-provider part generalizes to contracts that have associated dot providers, not just structure-type names. The main document says that, but I haven't written the extra document well enough, yet. Things can also be dot providers directly, but I haven't written that down well enough, either.

Parenthesization does not normally defeat dot providers. The "modulo parentheses" part at the top is meant to say that extra parentheses are fine, except currently in that last special case (which could be revised).

See the updated enforestation proposal (#162) for more on lexicons, which are used for imports, and why the .s are different. The rationale provides an example referring to weather.(***) and weather.(!!!) operators.

[sorawee]

• Typo: precdence
• expression_macro -> expr.macro, definition_macro -> defn.macro, binding_operator -> bind.operator, etc.
• As I understand, vals doesn't exist. values is still in use.
• I think it would make sense to provide operators from a module+ submodule, so that they can be imported unqualified. Other bindings are still encouraged to be imported qualified.

[mflatt]

The latest draft fills in the explanation of static information as a generalization of dot providers, and the explained features are now implemented. The implementation now requires a Racket snapshot dated 20210807 or later.

Looking forward, my plan is to add lists and general indexing of the form <expr> [<index-expr>] to the proposal and prototype. That should be more of the same, but it will finish taking advantage of the syntax that shrubbery notation makes available. Then, I expect pause on the prototype while we discuss whether this a promising direction to continue — that is, whether it's a question of fixing the details or starting over with a different direction.

[jeapostrophe]

Excellent

[mflatt]

The latest version adds lists, arrays, and maps.

Really, this version is an overhaul of the static-information and binding system, which has evolved into a even more Turnstile-like system of bidirectional flow. The “downward” flow is mostly constrained to binding space (as opposed to expressions), but the val definition form is still special: it's the one place where static information flows downward from expressions to bindings. Still, the new val is not so ad hoc and non-composable as it was previously.

In summary, the proposal is now really four things:

• an reader-level approach, which is shrubbery notation (#122);
• an expansion/parsing approach, which is enforestation via the Rhombus expander (#162);
• a static-information approach (which could be broken out into a separate proposal); and
• some specific syntactic forms, functions, and datatypes.

These four things are separable to a large degree. For example, the Rhombus expander and the static-information layer could be adapted to a different reader-level syntax. It makes sense to discuss the merits of individual pieces, and that's why there are multiple PRs. But these pieces have also been codesigned in an attempt to make everything fit together nicely, and it's probably easier to judge the combination than the pieces.

[jeapostrophe]

Ship it ;)

[mflatt]

The latest revision that introduces -: as an alternative to :: that does not imply a run-time check. Also, . is allowed to fall back to a dynamic lookup from a field name, instead of requiring a dot provider on the left, unless use_static_dot is used to define . as more strict.

These changes are intended to hit a better spot in the gradual-typing space. Although the intent is not for #lang rhombus to have a type system, trying to make . mean efficient field access is already a step in that direction... and we know that if you bridge static and dynamic worlds with dynamic checks, then it's easy to make things slow. Short of embracing a type system, the choices seem to be (1) stay away from static information, which means giving up certain constructs and idioms like . or relying on compiler improvements to make dynamic things fast; or (2) invest enough effort in the problem to avoid known pitfalls while supporting some gradularity in the initial design, at the risk of getting it wrong. The current proposal is (still) trying option 2.

[mflatt]

I've updated the proposal for changes to shrubbery notation (#122). Although the changes at the shrubbery level seem big, the effect on the #lang rhombus implementation and it examples seems small. The main change is to some macro protocols, as described in the discussion for #172. But {} is now available for use to mean sets or maps, if we want, and indentation without : can be used to continue a group on a new line starting with an operator.

[michaelballantyne]

The demo file reads much better for me with ~, ', and $.

[mflatt]

Just so everyone knows, the switch to ~ and ' and $ was based on discussion on the #rhombus channel on Discord (see the "Community" section near the bottom of https://racket-lang.org) with several people contributing.

Rhombus discussion should be here in GitHub as much as possible, especially on specific proposals, but this kind of detail benefited from real-time interaction. Of course, the conclusion is not set in stone, merely the next thing to try.

[sorawee]

Note that currently DrRacket seems to get frozen on #lang rhombus when an autocomplete plug-in is enabled. @yjqww6 provides a minimal program that triggers the problem:

#lang racket
(require syntax-color/module-lexer)

(define (symbols in)
  (let loop ([mode #f] [s (set)])
    (define-values (str type _1 _2 _3 _4 new-mode)
      (module-lexer in 0 mode))
    (cond
      [(eof-object? str) s]
      [(eq? type 'symbol)
       (loop new-mode (set-add s str))]
      [else (loop new-mode s)])))

(symbols (open-input-string "#lang rhombus\nfun"))

[mflatt]

@sorawee Change pushed to mflatt/shrubbery-rhombus-0.

[slaymaker1907]

Searchability of Constructs

It is probably a good idea to have names for uncommon operators in order to improve searchability. If a beginner sees fun flip(p -: Posn):, they might want to search for and figure out what -: is doing. By having an official name (maybe as an alternate identifier with the same programmatic meaning), DrRacket could show this name on hover. Since docs and Q&A sites would presumably use this name when explaining the concept, it would be better for SEO. Obviously Racket docs shouldn't have trouble searching for -:, but many people will just go straight to Google.

Formatting of RFC

I would recommend adding a TOC to the 0000-rhombus.md doc using internal links for easier navigation.

[sorawee]

I read

def Posn(pin_x, pin_y): pin
pin_x  // prints 3

and was very confused for a while, because I thought it's defining a function Posn. But even worse, one could do something like the following (though this goes against the convention that struct name should start with an uppercase letter):

struct posn(x, y)
def pin: posn(3, 4)
def posn(pin_x, pin_y): pin // function def or pattern matching?
def posn2(pin_x, pin_y): pin_x // function def or pattern matching?

I really like pattern matching though, so personally I think either of the following should happen:

1. Remove or redesign the function definition shorthand
2. Make the convention mandatory: enforce that the first character of struct name must be an uppercase letter.

For (1), I'm thinking about languages like ReasonML which does not provide function definition shorthand, but its lambda syntax (borrowed from JavaScript) is really concise. And users seem to be perfectly fine with the absence of the function definition shorthand.

let add = (x, y) => x + y;

For (2), it will still be confusing, but not as confusing as the def posn one.

[mflatt]

@sorawee Another possibility: get rid of def.

Using val or fun is unambiguous, while resolving def depends on binding in even more ways. We could avoid def, or we could rely on programmers choosing val or fun when it's not immediately obvious what def would do.

[michaelballantyne]

@mflatt could you elaborate on this bit from the proposal?

But having def also makes it easier to have let. A let modifier that could be applied to any definition form creates a lot of extra complexity, because definitions can expand to multiple bindings, and some of them need to refer to each other.

I much prefer val and fun to the overloaded def, but it might also be nice to support let val and let fun.

[mflatt]

@michaelballantyne

To make sure we're on the same page: It would work to have let val and let fun specifically. Having let any_definition_macro .... is trouble.

Suppose that let blindly expands the rest of the group as a definition, which in general produces a sequence of definitions, and then it adds scopes to defined identifiers to implement directed binding.

How would let know which right-hand sides get the extra scopes? For something like fun, the right-hand side should not get scopes, so it sees earlier definitions. For something like struct, the expansion involves definitions using the same symbolic name in different spaces plus some invisible (outside of expansion) names; some of the definitions are self-referential, and in at least once case the definitions are mutually referential, so binding each name so that it's visible only later would break the expansion.

In general, it seems like only the binding form can know how the expansion should interact with let, and so some cooperation is needed — and that would be the extra complexity for writing definition macros in general.

I note that the issue exists even within val/let due to binding macros. The current design forces binding macros to deal with this, and it's an explicit part of the low-level binding-macro protocol. But it would be nice to avoid that complexity for the definition-macro protocol, which is possible if it works to constrain let to a few likely useful cases like val and fun (whether because val or fun is inferred by let or because it's made explicit with let val or let fun).