pyinfra icon indicating copy to clipboard operation
pyinfra copied to clipboard

v3.0b0: variables passed to `@deploy(data_defaults)` not accessible via `host.data` in jinja2 templates

Open cmenke opened this issue 1 year ago • 0 comments

Describe the bug

Version: v3.0b0

When rendering jinja templates with the files.template()-operation, variables passed to @deploy(data_defaults) are not accessible via host.data.

This appears to be a regression from v2.9.

To Reproduce

Here's a simplified example:

# modules/test.py
from io import StringIO
from pyinfra.api import deploy
from pyinfra.operations import files

# foo is NOT defined in host/group data
DEFAULTS = {"foo": "bar"}

@deploy("Test", data_defaults=DEFAULTS)
def test_template():
    files.template(
        src=StringIO("foo={{ host.data.foo }}"),
        dest="/tmp/test"
    )
# task.py
import modules.test

modules.test.test_template()

Expected behavior

Running pyinfra inventory.py task.py works in v2.9 and writes "foo=bar" to /tmp/test. However in v3.0b0, an error is thrown:

jinja2.exceptions.UndefinedError: 'pyinfra.api.host.HostData object' has no attribute 'foo'

The full output can be found below.

Meta

pyinfra inventory.py task.py
--> Loading config...
--> Loading inventory...
    inventory.py is neither a valid Python import or hostname.

--> Connecting to hosts...
    [192.168.6.11] Connected

--> Preparing operations...
--> Preparing Operations...
    Loading: task.py
    [192.168.6.11] Ready: task.py

--> Detected changes:
    Operation                                                                                             Hosts                  
    Test | Files/Template (src=StringIO(hash=dbe452ba519ba243e393e2a5e45eb31fedce21df), dest=/tmp/test)   1/1 (192.168.6.11)   

    Detected changes displayed above, skip this step with -y
                             
--> Beginning operation run...
--> Starting operation: Test | Files/Template (src=StringIO(hash=dbe452ba519ba243e393e2a5e45eb31fedce21df), dest=/tmp/test)
Traceback (most recent call last):
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/pyinfra/operations/files.py", line 991, in template
    output = get_template(src).render(data)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
jinja2.exceptions.UndefinedError: 'pyinfra.api.host.HostData object' has no attribute 'foo'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/pyinfra/api/operations.py", line 185, in _run_host_op_with_context
    return run_host_op(state, host, op_hash)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/pyinfra/api/operations.py", line 53, in run_host_op
    return _run_host_op(state, host, op_hash)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/pyinfra/api/operations.py", line 82, in _run_host_op
    for command in op_data.command_generator():
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/pyinfra/api/operation.py", line 238, in command_generator
    for command in func(*args, **kwargs):
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/pyinfra/operations/files.py", line 1002, in template
    with open(src, "r") as f:
         ^^^^^^^^^^^^^^
TypeError: expected str, bytes or os.PathLike object, not StringIO
2024-02-14T11:06:16Z <Greenlet at 0x10c475800: _run_host_op_with_context(<pyinfra.api.state.State object at 0x10c2e77a0>, Host(192.168.6.11), 'ba63029b861efac1dc89396092183678302576e4')> failed with TypeError

--> Disconnecting from hosts...
--> An internal exception occurred:

  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/pyinfra/operations/files.py", line 1002, in template
    with open(src, "r") as f:
    ^^^^^^^^^^^^^^^^^
Traceback (most recent call last):
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/pyinfra/operations/files.py", line 991, in template
    output = get_template(src).render(data)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
jinja2.exceptions.UndefinedError: 'pyinfra.api.host.HostData object' has no attribute 'foo'

During handling of the above exception, another exception occurred:

TypeError: expected str, bytes or os.PathLike object, not StringIO

--> The full traceback has been written to pyinfra-debug.log
--> If this is unexpected please consider submitting a bug report on GitHub, for more information run `pyinfra --support`.
pyinfra --support
    If you are having issues with pyinfra or wish to make feature requests, please
    check out the GitHub issues at https://github.com/Fizzadar/pyinfra/issues .
    When adding an issue, be sure to include the following:

    System: Darwin
      Platform: macOS-14.2.1-x86_64-i386-64bit
      Release: 23.2.0
      Machine: x86_64
    pyinfra: v3.0b0
    Executable: /Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/bin/pyinfra
    Python: 3.12.1 (CPython, Clang 15.0.0 (clang-1500.1.0.2.5))

Please let me know if you need any additional information or details.

cmenke avatar Feb 14 '24 12:02 cmenke