(f-)string NG improvements plan

[plajjan]

• [ ] continue parsing expr etc using same parser so we don't start over and loose src loc
  • and expr parsing should be simpler in the sense that we don't have to deal with escaping quotes etc
• [ ] support escaping quotes e.g. f"hello \"thing\""
• [ ] implement exhaustive validation as basis for error messages
• [ ] error messages with good src loc, pointing to the most precise thing
• [x] no expensive ++ operations in parsing, cons + reverse plxz
• [ ] treat string as f-strings, i.e. make f prefix optional
  • we don't currently allow mixing % with f-strings, so changing normal strings "" to do f-string interpolation is backwards breaking because current "%s" % foo would break
  • thus, the plan is:
    • [ ] replace % with f-strings in all programs
    • [ ] in Acton, change normal strings to do f-string interpolation
    • [ ] remove f in all programs for idiomatic style "{foo}"
  • although we should allow f"" so we stay compatible with Python, just making it optional

[plajjan]

@nordlander I noticed that the current f-string implementation doesn't support mixing with %. While we did talk about making f-string interpolation the default, i.e. not require f"" but apply the same to "normal" strings "{hello}". Does that mean % should go away completely as a front-end kind of thing (we'd keep it internally as a translation target).

We currently do not support mixing in % inside of f-strings, like f"hello {foo} %s!" % bar. I didn't account for that, so the translation maps into something invalid I suppose. We could fix that, I'm just considering if we want to support that or if % style formatting should be removed entirely?

I guess from one perspective, it's fine to keep % and let people use what they prefer. On the other hand, it's easier with a simpler language...

[nordlander]

@plajjan I think % for strings should be hidden away. Our current support is really ad hoc, considering that our grammar actually sees % as a generic binary operator that can be overloaded via the Integral protocol. But when the left operand is a string literal, the type-checker instead applies a hard-coded rule that derives a type for the right operand by analyzing occurrences of % in the literal itself -- quite a breach with the type rule suggested by Integral!

The f-strings are a much more principled mechanism, that can be explained without referring to special type rules that contradict our built-in protocols. On the other hand, we do have a need for an internal representation of f-strings whose embedded expressions have been type-checked and extracted into code. So I suggest we keep the current mechanism (essentially the primitive operator FORMAT and its special handling in the code generator) just as a translation target, and remove the special rule for % on strings from the type-checker.