Quasistring.

[belmarca]

Please comment on the implementation.

Also make sure to squash, no need for all these commits.

[belmarca]

Docs coming in another commit.

[vyzo]

Summoning @fare for a second opinion.

[belmarca]

Here is one issue that needs to be adressed. The following does not fully work:

> (import :std/misc/text)
> (def t (template/hash "(2 + #{n}) is #{(+ 2 n)}"))
> (pp t)
(lambda (#0=#:ht44)
  (string-append
   "(2 + "
   (std/format#format #1="~a" (hash-get #0# 'n))
   ") is "
   (std/format#format #1# (+ 2 n))
   ""))

The macro does not parse the (+ 2 n) s-exp to properly reference n. Even with a datum->syntax this won't reference the proper variable, as the semantics of template/hash assume our references to be lookups in a hash table.

I will see about having a custom %%ref that, at expansion time, would turn (+ 2 n) into (+ 2 (hash-get ht 'n)).

[belmarca]

We will not be merging templates as their implementation is not considered ready yet.

[fare]

I didn't look too deeply into the implementation, but here's what bugs me about the design: you first parse the string as a string, then re-parse it. That's Scala-level lame. The right way to do it is more like what you have in a shell double-quote syntax, or in JavaScript backtick syntax: you parse only once, and can thus easily nest expressions with quasiquoted strings without horrible levels of escaping.

[belmarca]

Hi @fare,

Two things:

First, I am not at all surprised that it is suboptimal as this is the first ever parser I wrote. I just brute-forced my way out. I chose to open the string as a port to parse it one character at a time using read-char. The read-all-as-string procedure is used to parse sub-templates. If an expansion-time template variable expands into another template, we have to parse it. This is why I append the remaining quasistring to the newly expanded quasistring, and recurse. I would be glad to learn a thing or two on how to do this more cleanly.

Second, do you have an opinion on the semantics of the quasistring? Does something like

(def (qs name) (quasistring "Hello, #{name}!"))
(displayln (qs "Fare"))

look sound?

[fare]

When designing a syntax for that, it pays to think about existing systems that were or weren't successful at it. I believe the shell syntax, and to a lesser point, JavaScript syntax, are appropriate comparison points. Why not leverage shell syntax wholesale, for instance? "Constant part $(function call) ${var} \$escape" Alternatively, port Racket's scribble syntax (if the Racket code doesn't speak to you, try my CL code instead). Recursive syntax is MUCH better than parsing strings.

[belmarca]

Thanks @fare, I understand better your position.

What you are proposing is semantically different from what I have here. We could make a distinction between function calls and variables in the DSL. However I chose not to in order to keep the DSL to a strict minimum.

Instead of making the distinction between function calls and variables, I decided to make the distinction between phases. That is, #{} gets evaluated at run-time and ##{} at expansion-time. I don't know how I could get data computed and inserted inside a template at compile time otherwise. For the rest, we just let gerbil (well, %%ref and %%app) make the correct decisions wrt function application and reference.

For example, say I want to have my template reflect how long ago it was compiled. I can do

(import :std/misc/text)

(def (time-since then (now (time->seconds (current-time))))
  (inexact->exact (round (- now then))))

(def (qs name)
  (quasistring*
   "Hello, #{name}
This template was compiled ##{(let (then (time->seconds (current-time)))
(string-append \"#{(time-since \" (number->string then) \")}\"))} seconds ago"))


> (pp qs)
(lambda (#0=#:name56)
  (string-append
   "Hello, "
   (std/format#format #1="~a" #0#)
   "\nThis template was compiled "
   (std/format#format #1# (time-since 1572742630.4156442))
   " seconds ago"))
> (displayln (qs "world"))
Hello, world
This template was compiled 6 seconds ago
> (displayln (qs "world"))
Hello, world
This template was compiled 8 seconds ago
> (displayln (qs "world"))
Hello, world
This template was compiled 10 seconds ago
> (displayln (qs "world"))

which, I agree, has some rather clunky string escaping. This can be fixed three ways: with the use of "sub-quasistrings", user-defined functions/macros or lastly by augmenting the DSL with a way to define variables directly inside the template for further reference.

That last options would require quasistring to return a struct or class in which we could store variables. But that is beyond the scope of what I imagine quasistring to be, for the moment.

I didn't know about Scribble, but it seems to be a much larger DSL, which is precisely what I don't want to implement. I will go through their documentation but as I mentioned, I want quasistring to introduce as little new syntax as possible, letting the user work with gerbil and not a new DSL.

But I understand your point. I am willing to make changes if we can keep the run-time/expansion-time separation explicit. And anything that makes the implementation more maintainable and prettier is welcome.

[vyzo]

I kind of like the handlebars!

[belmarca]

I think quasistring fills a different need than scribble et al. One should be able to build a markup language by using quasistring and user defined macros. This is why I removed the template macros from this PR. Solving that problem is much more involved (and opinionated) and there are probably existing solutions that can be recycled, like Scribble.

[belmarca]

Ping @fare!

[fare]

I'm afraid of a 80% solution, when a 100% solution isn't far away: it's an attractive nuisance that only invites people to get into real bad situations.

[vyzo]

@fare why do you think it's an 80% solution and what's the 100% solution in that case? This is not intended to be the answer to all templating needs of the world, it is simply string interpolation.

Btw, I think we could reduce this and remove the compile-time evaluation stuff. If you need compile-time eval you can do it with a separate macro that does just that and create a value that is placed in the quasistring.

[belmarca]

I am currently using quasistring by compiling it into a separate package. I don't need it to be in the stdlib for now, we can definitely discuss this more. I also want to know what 20% you find missing.

I can extend this PR to have a quasistring procedure as well as a macro. My implementation is directed by my needs: I currently need compiled templates with nested templates which include both compile-time and run-time data. Hence what you see here.

[belmarca]

Closing. Neat idea but need to reimplement.