`require` fails when requiring a macro-utils file from other macro files with `compilerEnv` as `_G`

[datwaft]

The issue is that when the compiler macro option compilerEnv is set to _G, and you want to require a module that uses macro-exclusive functions from the macro file, neovim displays the following error:

.../packer/start/hotpot.nvim/fnl/hotpot/searcher/module.lua:113: Compile error in /root/.config/nvim/fnl/core/macro-utils.fnl:9
  symbols may only be used at compile time
        (= `fn (. x 1))
            ^^
            * Try moving this to inside a macro if you need to manipulate symbols/lists.

This could be the case if you want to share code between multiple macro files, and therefore you create an utils module.

If you don't use that compiler macro option or you don't use an utils module but have the code in the same macro file, everything works perfectly.

How to reproduce the issue

Note: you must have Docker installed on your computer.

Clone the following repository: datwaft/hotpot-issue_48

Execute the following terminal command:

docker build -t hotpot-issue . && docker run -it hotpot-issue

After that execute neovim:

nvim

Expected result

These lines should print "Hello"
================================
Hello
Hello
These lines should print "Bye"
==============================
Bye
Bye
Bye

Current result

The following error is displayed.

.../packer/start/hotpot.nvim/fnl/hotpot/searcher/module.lua:113: Compile error in /root/.config/nvim/fnl/core/macro-utils.fnl:9
  symbols may only be used at compile time
        (= `fn (. x 1))
            ^^
            * Try moving this to inside a macro if you need to manipulate symbols/lists.

[rktjmp]

That error is from Fennel directly, seems that it behaves the same outside of Hotpot.

--no-compiler-sandbox == compilerEnv = _G and --globals "*" == allowedGlobals = false from my interpretation of the docs.

~/pr/bu/hotpot-issue_48/co/fnl main ⚡
λ ./fennel --no-compiler-sandbox --globals "*" -c core/init.fnl
Compile error in core/init.fnl:7
  Compile error in ./core/macro-utils.fnl:9
  symbols may only be used at compile time

      (= `fn (. x 1))
          ^^
* Try moving this to inside a macro if you need to manipulate symbols/lists.
* Try using square brackets instead of parens to construct a table.
stack traceback:
	[C]: in function 'error'
	./fennel:3363: in function 'fennel.friend.assert-compile'
	./fennel:2211: in function 'fennel.compiler.assert'
	./fennel:3209: in function 'fennel.compiler.do-quote'
	(...tail calls...)
	./fennel:2681: in function <./fennel:2680>
	(...tail calls...)
	./fennel:1697: in function <./fennel:1692>
	(...tail calls...)
	./fennel:2681: in function <./fennel:2680>
	(...tail calls...)
	./fennel:1573: in function <./fennel:1568>
	(...tail calls...)
	./fennel:2681: in function <./fennel:2680>
	(...tail calls...)
	./fennel:1573: in function <./fennel:1568>
	(...tail calls...)
	...	(skipping 4 levels)
	[C]: in function 'require'
	./core/macro-module.fnl:1: in main chunk
	(...tail calls...)
	./fennel:2020: in field 'require-macros'
	src/fennel/macros.fnl:288: in function <src/fennel/macros.fnl:283>
	(...tail calls...)
	[C]: in function 'xpcall'
	./fennel:2657: in function 'fennel.compiler.macroexpand'
	./fennel:2837: in function 'fennel.compiler.compile1'
	./fennel:3068: in function 'fennel.compiler.compile-stream'
	(...tail calls...)
	[C]: in function 'xpcall'
	./fennel:5660: in function <./fennel:5646>
	(...tail calls...)
	[C]: in ?

(import-macros {: if-fn-hello-else-bye} :core.macro-module)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* Try moving this to inside a macro if you need to manipulate symbols/lists.
* Try using square brackets instead of parens to construct a table.

~/pr/bu/hotpot-issue_48/co/fnl main ⚡
λ

[datwaft]

Interesting, I will do some more testing and open an issue on the Fennel repo.

Thanks for you help!

[datwaft]

Opened the following issue on the Fennel repository: https://github.com/bakpakin/Fennel/issues/406

[datwaft]

https://github.com/bakpakin/Fennel/issues/406 is fixed now 🎉

[rktjmp]

I'll hold this open for now until I can make sure its fixed inside Hotpot too, have to review how we handle macro searching to make sure we are using the new code. I think it should be fine but it's been a few months. We run a bit of custom stuff to makes sure macro files are tracked as a dependency for cache invalidation which requires a one-line patch to fennel.lua to test.

[datwaft]

I was looking to check another issue and I saw that this issue was still open. At the moment it has worked perfectly so I think this issue can be closed.