Macros don't work as expected

[plexus]

Macros seem to actually be interpreted as regular functions, see

https://www.maria.cloud/gist/aee5dfdb9418c4f0ebc306369facc793?eval=true

Environment:

• Browser: Firefox, Chrome
• Platform: Linux

[mhuebert]

I'm glad you brought this up. I would really like a good defmacro experience, as the state of the art in cljs is rather complicated. I don't have a clear answer yet, but I like the idea of aiming for parity with how macros behave in Clojure.

Currently the way to get your example to work is: https://www.maria.cloud/gist/65446d9a1cbe9ae23a7d04580917c1fc?eval=true

ClojureScript normally creates macro namespaces from .clj or .cljc files, in a separate pass, creating a distinct namespace with $macros appended to the end of its name (eg. maria.user$macros). In Maria we don't think in 'files', only namespaces, so we need to switch to this $macros namespace for defmacro, then switch back to the non-macros namespace, and then refer in the macro, before we can use it.

Last year, I wrote a hacky attempt at a seamless defmacro-writing experience: https://github.com/braintripping/cljs-live/blob/716dec16cb52245e9ce260478b2d231be5e4e860/src/cljs_live/eval.cljs#L62

• Line 62, 64: detect defmacro form, & if so, use $macros namespace
• Line 75: after defmacro, refer the macro into the non-$macros namespace

Aside from being inelegant—I'd rather not stuff namespace surgery like this into eval—it was flawed because macros could not access functions defined in the non-$macros namespace. This could perhaps be 'fixed' with yet more namespace surgery, but I removed it entirely in the hopes that we could do it right at some point.

Maybe we could modify the compiler to resolve macros from non-$macros namespaces when in self-host mode? I'm not sure how involved such a change would be.

Additional thoughts...

• Something thing to think about, which may or may not be fixable via the same means as the general macro situation, is that syntax-quote in ClojureScript can be a pretty bad experience. When writing macros, I am constantly inserting ~' (unquote-quote) in front of variables because otherwise they resolve to things that the compiler can't find. This can make macros ugly to write and read. Again, parity with Clojure experience would be nice.

• The separate macros-pass in cljs has some weirdness, described in mfikes' blog:

  Even though the macros are defined in ClojureScript, they are defined in *.clj files. You can, if you wish, also define macros in *.cljc files, but when they are processed, the :cljs branch of reader conditionals will be used.

  So (1) you can define macros written in and for ClojureScript in a .clj file, but (2) if you move that macro to a .cljc file, you must put it in a :cljs reader-conditional branch. This seems like a misuse of the .clj file extension. Instead of meaning "this is clojure code", it is being used to say "this is a macros namespace, can be Clojure or ClojureScript code". It is confusing, at least to me.

[plexus]

Currently the way to get your example to work is: https://www.maria.cloud/gist/65446d9a1cbe9ae23a7d04580917c1fc?eval=true

Thanks, that's helpful!

Maybe we could modify the compiler to resolve macros from non-$macros namespaces when in self-host mode? I'm not sure how involved such a change would be.

This sounds like there should be an issue for this in ClojureScript JIRA, if there isn't already, just to see what the upstream sentiments are about this. I'm sure Lumo and Planck would also be interested in a more seamless macro experience.

This seems like a misuse of the .clj file extension. Instead of meaning "this is clojure code", it is being used to say "this is a macros namespace, can be Clojure or ClojureScript code".

I agree that this seems wrong, and again a conversation that perhaps we should be having with upstream.

[mhuebert]

Current stance of upstream:

https://clojurians.slack.com/archives/cljs-dev/p1507552339000103

Points to second-last item here:

https://github.com/clojure/clojurescript/wiki/Bootstrapped-ClojureScript-FAQ#with-bootstrapped-clojurescript-can-we-now-have-namespaces-that-mix-both-macros-and-functions

[mhuebert]

@thheller has got a great little patch here to work:

https://www.maria.cloud/gist/e9f1ef4f77f8598a0cce7c07f0d6be7b?eval=true

We can run behind the scenes, and then macros should 'just work'.*

*this makes the project uncompilable by the regular JVM compiler.

[jackrusher]

I'm a bit nervous about anything that breaks compatibility with the usual tool chain.

[thheller]

The change should only be applied at runtime so no tool should be affected.

You could define the get-expander* fn in some namespace like maria.eval. Then (set! js/cljs.analyzer.get-expander* patched-get-expander) before calling eval and set it back after (if necessary).

[mhuebert]

I am of two minds.

It would be consistent with the approach we've taken so far with curriculum to teach this easier, more rational style of macros (which is also how Clojure macros work) first. Once you understand what macros are and why you want to use them, then you may be ready to learn about how macros compile in multiple passes in ClojureScript, so you have to create multiple namespaces, have them reference each other, choose correct file extensions, deal with half-broken syntax quote. (that's all incidental complexity which has nothing to do with the already-challenging concept of macros.)

But the fact that existing tools can't compile these macros would mean that we could no longer spit the output of any arbitrary Maria program to a file and compile it using existing tools. (currently this is quite simple if you provide the right dependencies.)

...which implies that we also couldn't do advanced-compile (or any non-self-host compile) of Maria examples that use macros in this way. (We currently have a share button that allows users to link to an advanced-compile version of any form in a gist; it is compiled on a server somewhere using ordinary tools. We wouldn't want examples that contain macros to fail.)

[jackrusher]

I totally agree with wanting macros to work in a sane way, and the weirdness of macros under CLJS is one of the many irritations I feel when using it. But I think that we might want to push on this upstream for awhile to see if we can get it fixed in general, rather than breaking compatibility right away.

(@thheller See @mhuebert 's comment for what I meant about compatibility. 😸)

[thheller]

I don't think you can get it fixed upstream. The design decision to keep them separate was made with good reason which hasn't changed. You could fix it for self-host but then you'd end up with code that only compiles in self-host but not the JVM compiler, which is way worse.

An overarching design goal of bootstrapped ClojureScript is that namespaces be uniformly compilable, regardless of whether they are being compiled by the JVM-based implementation or when compiled in bootstrapped mode. Thus, bootstrapped ClojureScript intentionally preserves the constraints that have always been present.

I do think it is a good idea to make an exception for teaching tools like Maria to ease into macros since they are hard enough without all the limitations. So far I haven't seen Maria deal with namespaces, files or dependencies in general. Maybe if that is well integrated into the UI the fact that macros need to be in a separate files becomes less of an issue?

[mhuebert]

Several months after this thread was started, Mike Files released a custom defmacro which looks like it would suit our pedagogic purposes: https://github.com/mfikes/chivorcam

It would be nice to add this + an intro to macros to Maria.

[jackrusher]

Note that sci supports CLJ-style macros, so if we move away from self-hosted we won't need an additional dep.