Recursive macro or plugin causes obscure Nailgun client error

[gatesn]

I accidentally made my macro recursive:

pants-plugins/macros.py

def my_library(**kwargs):
    kwargs["tags"] = kwargs.get("tags", []) + ["sometag"]
    my_library(**kwargs)  # instead of python_library

pants.toml

...
build_file_prelude_globs = ["pants-plugins/macros.py"]
...

Resulting in:

09:16:52.16 [ERROR] Nailgun client error: "Nailgun client error: Client exited before the server\'s result could be returned."
Traceback (most recent call last):
  File "/home/ngates/.pex/installed_wheels/9c1d05a38e4354df4f0ecd5c09e6d2c483b486ea/pantsbuild.pants-2.2.0+git81fd0314-cp37-cp37m-linux_x86_64.whl/pants/bin/pants_exe.py", line 36, in main
    exit_code = runner.run(start_time)
  File "/home/ngates/.pex/installed_wheels/9c1d05a38e4354df4f0ecd5c09e6d2c483b486ea/pantsbuild.pants-2.2.0+git81fd0314-cp37-cp37m-linux_x86_64.whl/pants/bin/pants_runner.py", line 77, in run
    return remote_runner.run()
  File "/home/ngates/.pex/installed_wheels/9c1d05a38e4354df4f0ecd5c09e6d2c483b486ea/pantsbuild.pants-2.2.0+git81fd0314-cp37-cp37m-linux_x86_64.whl/pants/bin/remote_pants_runner.py", line 100, in run
    return self._connect_and_execute(pantsd_handle)
  File "/home/ngates/.pex/installed_wheels/9c1d05a38e4354df4f0ecd5c09e6d2c483b486ea/pantsbuild.pants-2.2.0+git81fd0314-cp37-cp37m-linux_x86_64.whl/pants/bin/remote_pants_runner.py", line 135, in _connect_and_execute
    command, args, modified_env
native_engine.NailgunClientException: Nailgun client error: "Nailgun client error: Client exited before the server\'s result could be returned."

I couldn't find any way to get logs from Nailgun server. Nothing obvious shows up even with -ldebug --log-show-rust-3rdparty

[Eric-Arellano]

I reproduce. FYI the nailgun error is coming from Pantsd. The error is even more ominous without Pantsd:

❯ ./pants --no-pantsd list ::
[1]    39912 illegal hardware instruction  ./pants --no-pantsd list ::

I'm not sure there's going to be much of a solution here. We could maybe try banning recursion in macro files, although that sounds non-trivial to detect and recursion could be used safely.

Probably worth investigating why we're getting an illegal hardware instruction, though.

[stuhood]

Did anything go into the pantsd log at .pants.d/pantsd/pantsd.log?

[stuhood]

It looks like the opaqueness of this error is due to setting Python's recursion limit too high (which we did in #11271). If you lower the recursion limit by setting PANTS_RECURSION_LIMIT=1000, you'll actually get a stacktrace.

I'll see how much of the reasoning from #11271 is still valid.

[github-actions[bot]]

This issue has been open for over one year without activity and is not labeled as a bug. It has been labeled as stale to invite any further updates. If you can confirm whether the issue is still applicable to the latest version of Pants, appears in other contexts, or its priority has changed, please let us know. Please feel free to close this issue if it is no longer relevant.