rhombus-prototype

rhombus-prototype copied to clipboard

is this right?

• familiar function notation: function(arg1, arg2, ...)
• infix notation: width*height
• brackets for grouping: (a + b)

I'd say it was parentheses for grouping, i.e. you can always put in parens and it doesn't turn things into some other mode

I think the three things were:

• infix notation: a + b with spaces like Pyret
• algebra-style function notation: function(arg, ...)
• parens for grouping: (a + b) where wrapping an expr in parens doesn't mean an extra function call

Would parentheses still be used to delimit individual expressions?

@dedbox My interpretation of Matthew is that you can always add ()s, unlike some non-() Sexprs where adding ()s drastically changes what something means

It's a minor point, but I don't think grouping parens "always" work in languages that have them. Some only let you add them around expressions.

Grouping parens serve the purpose of disambiguating expressions that use ambiguous (or in any case easily misunderstood) combinations of syntactic extensions. Therefore, I think an infix language with as much syntactic extension as #lang racket will likely find a need for grouping parens around each of the extensible grammar nonterminals: Expressions; assignment targets; match and syntax/parse patterns; require and provide clauses; and anything else I'm overlooking here. (Rename transformers? syntax/parse ellipsis-head patterns?)

On the other hand, maybe infix syntax doesn't have all the same benefits in every one of those places. Although infix syntax and grouping parens are familiar from the way people learn to compute with desktop calculators, I haven't heard of people punching in require clauses into those calculators, so there may not be much familiarity benefit to using infix there.

Edit: At first I talked about "anywhere," but then I realized the word you used was "always," so I patched it up to refer to that instead.

With respect to function calls, one of the things I love about Scheme is:

((get-my-fun a b) c d)

So, hopefully we'd still be able to do this. For example: (get-my-fun(a, b))(c, d) or get-my-fun(a, b)(c, d)

Is that an expectation shared by others?

I like that the idea that adding parenthesis doesn't change meaning ( Thank you @jeapostrophe ) this means that both have the same meaning; (get-my-fun(a, b))(c, d) get-my-fun(a, b)(c, d) or (adopting scribble at-exp notation) (@get-my-fun(a, b))(c, d) @get-my-fun(a, b)(c, d)

That matches my expectations @lojic --- In particular, both of those do what you think in C, which is obviously the best place to get advice from.

Could/should we achieve point (2), algebra-style function notation function(arg, ...), the same way Haskell does with tuples?

Make function expr1 expr2 ... mean function application of multiple arguments, so naturally grouping it with (function expr1 expr2 ...) would be just as valid Make function (expr1, expr2, ...) mean a function application passing arguments in a single tuple

I do like how Haskell handles this i.e. functions either being defined to accept a tuple, or to be defined as curried functions which can be partially applied.

Although Matthew seemed to have a strong opinion about including the commas. I also wonder if supporting the two approaches, especially in a dynamically typed language, may add confusion to the folks we're trying to appeal to. In other words, having folks differentiate between add x y z and add (x, y, z), and knowing when one is preferable to the other.

IIRC in Haskell, add x y z is equivalent to ((add x) y) z - would we also support that grouping?

No, I don’t think auto-currying function application like ((add x) y) z is a good idea for Racket since Racket has variable-arity functions.

add x y z should mean the same thing (add x y z) does now

I misunderstood your suggestion. If we're not allowing curried function definitions, then it seems neither add x y z nor (add x y z) are keeping with the spirit of providing a syntax that people are used to from middle school algebra i.e. won't they be expecting f(x, y, z) ?

They will be able to write f(x, y, z) because they will be able to write f arg with arg as a tuple (x, y, z), if f is defined to accept a tuple. However, add x y z looks better in many situations where x, y, or z is a complex expression that spans multiple lines, and also works better with existing (add x y z)

It sounds like that would lead to some friction in the module ecosystem, where some modules' functions would expect tuples and some would expect multiple arguments.

Sometimes fragmentation like that is inevitable if different languages have vastly different notions of what functions do. But I think if we want Racket 1 and Racket 2 to use mostly unified docs and to have mostly seamless interop, then if we do end up going for a syntax resembling func(a, b, c) in Racket 2, it should have the semantics Racket 1 functions expect.

(I've got a lot of ifs in there, because a lot of the questions of what Racket 2 is for, what kind of syntax it will have, and what role Racket 1 will play in the future are still subject to discussion.)

One of the choices in an overall syntax is how to handle grouping. Many alternative notations for Lisps involve the same sort of grouping as parentheses, but written some other way (e.g., newlines, a : to indicate that parens wrap the following terms).

Elixir and Honu (and maybe other languages) move away from that. Syntactic transformsations are still triggered by a leading identifier, but the macro implementing the syntactic form consumes some number of terms that follow. In other words, you have to know some bindings to know where a grouping ends. In Elixir, it seems that the number of terms consumed by a macro is fixed statically, and do: ... end is used as an alternative to parentheses or braces to group as needed into a single term for a macro. Honu does not have that constraint.

In the spirit of brainstorming as Neil suggested on the mailing list, here's another idea along those lines. It fits within the mechanisms of Honu, but it may have a more precedent that I haven't yet seen.

Idea: Instead of grouping terms by start and end tokens, group them by separator between the pieces that should be joined.

One concrete instantiation of that idea is to use & as separator for "begin"-like and "and"-like sequences and | as a separator for "match"-like and "or"-like sequences.

Some examples that show how Racket-style things might be written in a language that uses separators:

  (define (f s)
    (printf "hi ~a\n" s)
    "hi")
 =>
  ;; Sequence is less implicit and directly supported by `define`,
  ;; which continues when it sees `&`
  define f(s) =
    printf("hi ~a\n", s)
  & "hi"

  (cond
    [(one?) "one"]
    [(two?) "two"]
    [else "many"]
 =>
  ;; A sequence of `cond` clauses is "or"-ish, and
  ;; we neeed a `|` even on the first clause if we
  ;; want to allow 0 clauses
  cond e
  | is_one() "one"
  | is_two() "two"
  | else "many"

  (and (f) (g) (h))
 =>?
  ;; Looks redundant to have both `and` and `&`, but
  ;; maybe worth the consistency. Skipping a leading
  ;; `&` means that `and` needs at least one subform.
  and f() & g() & h()
 =>?
  ;; Maybe it's better to just use `&&` as an operator, anyway
  f() && g() && h()
 
  (or (f) (g) (h))
 =>?
  or f() | g() | h()
 =>?
  f() || g() || h()
 
  (let ([x 1]
        [y 2]
        [z 3])
     (+ x y z))
 =>?
  ;; A sequence of bindings seems "and"-ish
  let x = 1
    & y = 2
    & z = 3
    x+y+z
 =>?
  ;; Or is the "=" close enough to a separator already?
  let x = 1
      y = 2
      z = 3
    x+y+z

  ;; Example recently posted on Slack
  (define (powerset s)
    (generator ()
               (cond [(null? s) (yield '())]
                     [else
                      (for ([e (in-producer (powerset (cdr s)) (void))])
                        (yield (cons (car s) e))
                        (yield e))])
               (void)))
 =>
  ;; Note that parentheses are needed around `for` so that `& yield(e)`
  ;; belongs to the `for` body while `& void()` belongs to `generator`.
  define powerset(s) =
    generator ()
      cond
      | is_empty(s) yield(empty)
      | else (for e = in_producer(powerset(rest(s)))
                 yield(cons(first(s), e))
               & yield(e))
    & void()

Just to emphasize: relying on separators this way does not lead to an automatic parenthesization independent of binding, because the transformation would require knowing that match is bound to a syntactic form that uses | as a separator after the first form, and so on.

In Elixir, Remix, and Honu, there are a number of syntactic patterns that are effectively built into the reader layer. The reader converts patterns like infix syntax, dot notations, function calls into S-expressions that effectively record the original shape. See https://elixir-lang.org/getting-started/meta/quote-and-unquote.html for Elixir.

I forget exactly what Remix and Honu do, but you could imagine that <expr>(<expr>, ...) is grouped by the reader into #%app, <expr>[<expr>, ...] is grouped by the reader into #%index, <expr>.<expr> is grouped by the reader into #%dot, and so on. To avoid bad puns, instead of identifiers like #%dot that are potentially written literally in the source, the S-expression encoding could take a form that cannot be represented in the source except by actually using <expr>.<expr>.

I find this direction appealing because, like S-expressions, it provides a shared vocabulary for constructing terms without necessarily imposing a meaning on the terms. I also like the way it tends to distinguish syntactic form names from variable names. By that last statement, I have in mind the way that function application is a kind of fallback interpretation of parentheses in Lisp/Scheme/Racket. The fallback interpretation allows function calls to have a lighter syntax instead of putting #%app (or something shorter) on every application, but it makes it harder for someone approaching a new set of syntactic extensions to know which identifiers correspond to syntactic forms are which are just expressions that have a function value.

That idea is implicit in the examples I wrote in the previous comment. You have surely assumed that is_one is a function and not a syntatic form. In fact, is_one might be a macro, maybe because it is bound to a function with a contract — but if the author of is_one has any taste, then it's going to behave as an expression. Meanwhile, there's no question that match or generator was meant as a syntactic form.

One issue seems to be that finding the extent of an expression seems to require predicting the shape associated with identifiers inside the expression. That seems like a problem for local bindings forms (especially let-syntax but even let and lambda) and a bigger problem for a hypothetical where-like binding form.

1. The problem with let-syntax:

cond | one() let-syntax m = m_transformer() in m 1 | two() 2 | else 3

Does the two() 2 belong to cond or to m? What if m was named case? It seems that in order for cond to determine the extent of its first clause, it must predict the shape that m gets from m_transformer.

2. The problem with let:

cond | one() let case = add1 in case 1 | two() 2 | else 3

If we assume that case originally has a cond-like shape (except an expression before the first |), then when let shadows case, it should have the default variable/function shape instead. This is easier than (1), if let always binds names with the default variable/function shape, but it demonstrates that just eliminating let-syntax doesn't solve the problem.

3. The problem with where: It has the same problem, except the uses occur before the binding, so we probably can't even look at the binding to determine its new shape.

One solution might be to disallow local shadowings that change the shape of an identifier (where unbound identifiers have a default shape compatible with variable references and maybe function calls).

How did Honu deal with that?

@rmculpepper Great questions. For 1 and 2, I think probably we didn't notice that problem, because we didn't get to a concrete syntax for macro definitions in Honu (so we didn't get to the question of local macros. One possible answer is that parentheses are needed around the let body in that case. For 3, if I recall correctly, Honu couldn't handle where at all, since it stuck with macro dispatch based on a leading identifier (and trying to treat where as an infix operator would not work in Honu, I think).

I really like the Elixer/Remix/Honu principle of identifying a set of patterns that get special treatment in the reader. I like the creative thinking of @mflatt wrt & and |. I want to see us come up with a small set of such things that add up to a buttery C-like syntax.

When I look at beautiful, well-formatted C, I read it like my dream language where

• Macro applications start with a leading identifier, like while, switch, and typedef and consume units until a ;. (C doesn't enforce this, but I also write my C code with these terminals ;s.)
• Function and variable definition is so important that #%module-begin is made up purely of these definitions, plus module imports.
• Variable definition is very important in #%block, so that it starts with a sequence of =s that are parsed as defining variables and then switches to something else. return and == help to differentiate definition from expressions.
• Inside of parens and blocks, ; and , serve to continue the sequence, depending on what kind of sequence it is. , are for "expression sequences" and ; are for "statement sequences".
• . and -> are special infix operators that are bound to macros

What I tried to do a little bit with Remix and want to push more in Racket 2 is to have code that looks almost exactly like C, but has a little bit more regularity and the full power of an amazing Racket-style macro system.

I just posted a proposal in #88

It's great to see a concrete proposal to help discussion!

I remain torn between the alternatives of grouping in a binding-insensitive way (e.g., always until a semi-colon, always parentheses, always by indentation) and the grouping by enforestation approach of Honu. The former is simpler, but it tends to be more noisy, at least in the context examples I've seen. The latter is more flexible and can support cleaner looking syntax (IMO), but it requires more tool support.[*] Maybe more concrete proposals will help.

Although I'm not inherently opposed to a syntax that looks like C, one of the ideas that I found found most interesting in the discussion at RacketCon was from John Clements: we should consider picking a syntax for Racket2 that is intentionally distinct, so that when people see they code, they instantly recognize it as Racket2. Just as a straw man, swapping the role of curly braces and square brackets compared to C could have that effect, and it could also lesson the discomfort of semi-colons that look unnecessary to eyes trained to read C. Of course, a swap like that could also look gratuitous.

[*] When I think about tool support, it reminds me that we want better indentation support in DrRacket, and maybe that's essentially the same problem as dealing with enforestation. Even if so, concluding that we need certain tool support in DrRacket is not the same as concluding that it's an acceptable requirement for all parts of the ecosystem.

I don't think I can say anything new about the virtue or not of binding-insensitive grouping. I agree with you on the advantages and disadvantages. I am very pessimistic about good tool support and think it is very important to easily support a large number of editors on day 1.

As far as the distinctiveness, I agree that something distinctive would be nice. I think it would very gratuitous to base it on search&replace. I am not creative enough right now to think of something radically different.

However, I think that C-expressions can lead to very distinct styles. Here are five programs that I believe are all C-expressions and are also real programs in five different languages. I may have added parens or other little tokens not found in the wild, but I bet you could guess what language each is.

If you can, then you should believe that C-expression-based languages are not all the same; in the same way that it is very clear if a program is written in R5RS, Racket, Common Lisp, or Clojure. Here are the five programs:

Example 1:

const express = require("express");
const app = express();
const port = 3000;

app.get("/", (req, res) => res.send("Hello World!"));

app.listen(port, () => console.log("Example app listening on port ", port, "!"));

app.get("/", function (req, res) {
  res.send("Hello World!");
});

Example 2:

int main () {
  int i;
  const char* s[] = { "%d\n", "Fizz\n", s[3] + 4, "FizzBuzz\n" };
  for (i = 1; i <= 100; i++)
    printf(s[(!(i % 3)) + (2 * (!(i % 5)))], i);
  return 0; };

Example 3:

int main() {
 int a[] = { 1, 3, -5 };
 int b[] = { 4, -2, -1 };
 
 std::cout 
  << std::inner_product(a, (a + sizeof(a)) / sizeof(a[0]), b, 0)
  << std::endl;
 
 return 0; };

Example 4:

public final class FlattenUtil {

 public static List<Object> flatten(List<?> : list) {
  List<Object> : retVal = new LinkedList<Object>();
  flatten(list, retVal);
  return retVal; };
 
 public static void flatten(List<?> : fromTreeList, List<Object> : toFlatList) {
  for (Object : item = fromTreeList) {
   if (item `instanceof` List<?>) {
    flatten(List<?> : item, toFlatList);
   } else {
    toFlatList.add(item); }; }; }; };

Example 5:

fn factorial_recursive (n: u64) -> u64 {
 match n {
  0 => 1,
  _ => n * factorial_recursive(n-1) }; };
 
fn factorial_iterative(n: u64) -> u64 {
    (1..n+1).fold(1, |p, n| p*n); };
 
fn main () {
    for i in 1..10 {
        println!("{}", factorial_recursive(i)); };
    for i in 1..10 {
        println!("{}", factorial_iterative(i)); }; };

The first thing that jumps out at me when I look at those examples is a fair number of patterns of alternating ; and }, somewhat reminiscent of the profusion of closing parens we have now. That got me thinking that both c-expressions and s-expressions make sense to people who are familiar with those types of syntax, but can seem quirky to newcomers. From there, it occurred to me that one of the most commonly encountered differences between programming languages and natural languages is that natural languages generally use periods to end statements. Maybe that could be a "distinctive feature" of the syntax? More generally, perhaps a syntax that takes relatively more cues from natural language conventions than is typical?

Some of these exchanges also got me thinking on a slightly more meta/wants kind of level:

Are we including lowering barriers for "potential programmers" or "first-time programmers" when we aim for "programmers generally"?

Hearing the tradeoffs about grouping (more flexible, cleaner, dependent on tooling), would it be helpful to start work towards compiling a list of criteria that we use to communicate about decisions? I don't think we're at the stage where we could put together a discussion framework per se, but maybe writing down in a central location some of the ad hoc things people have mentioned would help? Kind of a "checklist of things to consider" or some such, which would of course be very fluid right now.

(Yes, I am volunteering to kick off or contribute to some of these threads if they are deemed helpful and no one more qualified has the time.)

@bzyeung I think you should make such a thing.

I agree that }; pattern is ugly. I think that's what Matthew and I meant by "noisy". I think it is actually there because the C-style is not normally functional so it looks odd to have so much nesting. It's definitely a trade-off to get the tree-shape. I think there are a lot of little things that could be done taking a cue from some of the S-expression alternate notations.

For example, we could use a symbol like MATHEMATICAL RIGHT WHITE TORTOISE SHELL BRACKET ⟭ as a "giant right parenthesis" that closes everything that is open. If that's too absurdly Unicode, then taking a cue from you, a . as the last non-comment character on a line could mean the same thing. I think it would be useful inside of functions, but not really inside of "classes" because you may have more functions afterwards. So the "Rust" example would look like:

fn factorial_recursive (n: u64) -> u64 {
 match n {
  0 => 1,
  _ => n * factorial_recursive(n-1).
 
fn factorial_iterative(n: u64) -> u64 {
    (1..n+1).fold(1, |p, n| p*n).
 
fn main () {
    for i in 1..10 {
        println!("{}", factorial_recursive(i)); };
    for i in 1..10 {
        println!("{}", factorial_iterative(i)).

But the "Java" example is

public final class FlattenUtil {

 public static List<Object> flatten(List<?> : list) {
  List<Object> : retVal = new LinkedList<Object>();
  flatten(list, retVal);
  return retVal; }; // <--- No .
 
 public static void flatten(List<?> : fromTreeList, List<Object> : toFlatList) {
  for (Object : item = fromTreeList) {
   if (item `instanceof` List<?>) {
    flatten(List<?> : item, toFlatList);
   } else {
    toFlatList.add(item).

A similar idea would be to have a combination of indentation and newlines be relevant: An indented line would imply a { and a blank line would imply a };. So, the "Java" version could look like:

public final class FlattenUtil
 public static List<Object> flatten(List<?> : list)
  List<Object> : retVal = new LinkedList<Object>();
  flatten(list, retVal);
  return retVal;
 
 public static void flatten(List<?> : fromTreeList, List<Object> : toFlatList)
  for (Object : item = fromTreeList)
   if (item `instanceof` List<?>) {
    flatten(List<?> : item, toFlatList); }
   else {
    toFlatList.add(item); };
// close for

// close flatten

// close class

It's a bit awkward to have those three blank lines at the end, so I think stylistically, we'd encourage people to include the }; for the for and class.

I don't think I'm creative to come up with a new syntax. I really like tree shapes and I really like the different notational textures in C-style code. When I actually program in C, I am annoyed that it is not more tree like. In Racket, I am annoyed it is so uniform. When I program in Haskell and ML, I am really annoyed at how not tree-like and fiddly they are. (I find the Haskell indentation rules excruciating.)

Another brainstorming variant: use separators for grouping — but do that always, so there's no dependence on binding to group terms.

https://gist.github.com/mflatt/f7313e1561210be56313903fd75c6607

As you'll immediately notice, this direction seems to need precedence(!) to work nicely for simple examples. And for complex examples, like the one at the end, just use parentheses when the precedence gets too hard.

We may want to reserve colon for a more type-like meaning, so maybe it's not a good choice for the operator to use after a syntactic form and it's distinct kinds of parts. While the : immediate after a syntactic form has its appeal, the extra : to separate let bindings from the body or a match expression from the clauses is less appealing.

Something that I'm wondering is if it is good that the first clause in a cond is special. For example instead of:

cond :
  is_one() => "one"                               // <-- without "|"
| is_two() => "two"
| else => printf("else\n"), "many"

one possibility is:

cond :
| is_one() => "one"                               // <-- with "|"
| is_two() => "two"
| else => printf("else\n"), "many"

It's almost like the indentation in Python. It makes the (multiline) expression more uniform and makes reordering and adding/removing clauses more easy.

It's also more similar to (my) programming in whiteboard where the blocks are marked with a big line at the left margin.

(Also, I've heard some complains about Erlang, because in some cases you have to use : or ; or . or , and when you reorder the expressions you have to change the symbol. I never used it to be sure that how painful is it and if the complain is about a real problem.)

@mflatt I don't find it beautiful, but I think it works. I don't mind precedence in general (my C-exprs have them) but it can be confusing to have too many. Different things and I worry as well that if we pick 7 things, then in X times we'll wish we picked 9 or 6. It will really come down to how we tend to write the new forms. I think we'd want ; included on the right and . included at the left.

In your examples, I don't understand if you can tell the difference between different parenthesizations. For example f x , y is (, (f x) y) and f (x, y) is (f (, x y)) and (f (x, y)) is ((f (, x y))? The only one that seems weird is the last one, but I don't know why it isn't that.

You're also not using [] and {}. I assume that we'll support those and they will be different than ()? (and not just with an easy to ignore paren-shape property.)

@gus-massa I agree with you that the extra | is nice. I get hurt by that in Datalog and Haskell and Coq all the time. However, I don't think that is a syntax notation question, but a question about how we should use this notation to write the cond macro. We could have the same concern about the ELisp cond (no parens around question-answer pairs and the tail is the else) versus the Racket cond (brackets around the pairs and the else still gets a question.) --- @mflatt IMHO, the question of what to use : for is like this.

Part of this experiment was imagining a distinction between "statement" grouping an "expression" grouping. I had in mind leaving . as an expression-level operator like + and * and &&. Maybe that's equivalent to saying that all expression operators have higher precedence, but part of what makes the notation work for me is that f(x, y) is parsed as an expression, so I'm not sure it's just an extension of precedence. No opinion on ;.

I had in mind that f(x, y) is a function-call form, and f (x, y) is two separate forms. In other words, whitespace would matter for parsing a function-call form. f x , y is two forms, f and x, y, with no function-call forms in it. Along similar lines, I had in mind parsing [] with no whitespace before as indexing. I had no ideas beyond that for [] and {}.

I went back and forth on whether : or | made more sense for cond. I decided to try the convention of using : to separate a syntax form's name from its parts, but I agree that | looks better for cond and for match after the first expression. Although | looks better for define with multiple function cases, it looked strange to me in define with one case.

I like to imagine how the final language would look like. A sane syntax is important but a nice snippet too.

Is it possible to make the first | optional and make the reader magically fix it? For code in a single line the first | is not nice. (And I think it's better not to look at the \n to decide how to put the parenthesis.)

cond : x => 1 | else => 2
cond : | x => 1 | else => 2      // :(
cond : 
  x  => 1      // :(
| else => 2
cond : 
| x  => 1
| else => 2

How do I nest a cond and a match?

cond : x => match : v : "y" => 1 | "z" => 2 | else => 3    // ???
cond : x => (match : v : "y" => 1 | "z" => 2) | else => 3

What about the => numerical operator? (I guess it will not be a problem, because it is already "overloaded" and it is not a problem.)

printf(x => y)