deepcopy(jinja2.Template(...)) raises an exception

[OddBloke]

MCVE

from copy import deepcopy

from jinja2 import Template

deepcopy(Template(''))

Expected Behavior

A deep copy of the Template object is returned.

Actual Behavior

A TypeError is raised.

Full Traceback

Traceback (most recent call last):
  File "copy_example.py", line 5, in <module>
    deepcopy(Template(''))
  File "/home/daniel/.virtualenvs/jinja/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/daniel/.virtualenvs/jinja/lib/python3.6/copy.py", line 274, in _reconstruct
    y = func(*args)
  File "/home/daniel/.virtualenvs/jinja/lib/python3.6/copyreg.py", line 88, in __newobj__
    return cls.__new__(cls, *args)
TypeError: __new__() missing 1 required positional argument: 'source'

Your Environment

• Python version: Python 3.6.2
• Jinja version: 2.9.6 and git master

[OddBloke]

Playing around with this, the following seems to work in trivial cases:

diff --git a/jinja2/environment.py b/jinja2/environment.py
index 2a4d3d7..1f8f309 100644
--- a/jinja2/environment.py
+++ b/jinja2/environment.py
@@ -11,6 +11,7 @@
 import os
 import sys
 import weakref
+from copy import deepcopy
 from functools import reduce, partial
 from jinja2 import nodes
 from jinja2.defaults import BLOCK_START_STRING, \
@@ -944,6 +954,18 @@ class Template(object):
             None, 0, False, None, enable_async)
         return env.from_string(source, template_class=cls)
 
+    def __deepcopy__(self, memo):
+        copy_namespace = {
+            'name': self.name,
+            '__file__': self.filename,
+            'blocks': self.blocks,
+            'root': self.root_render_func,
+            'debug_info': self._debug_info,
+        }
+        return self._from_namespace(
+            deepcopy(self.environment), deepcopy(copy_namespace),
+            deepcopy(self.globals))
+
     @classmethod
     def from_code(cls, environment, code, globals, uptodate=None):
         """Creates a template object from compiled code and the globals.  This

However, root_render_func is generated with a hard-coded reference to the environment that is generating it which means that copied templates will still contain a reference to the environment of the source template.

This means that the following code doesn't error with an UndefinedError on the last line:

from copy import deepcopy

from jinja2 import StrictUndefined, Template

t1 = Template("{{ foo }} {{ bar }}")
t2 = deepcopy(t1)
t2.environment.undefined = StrictUndefined

kwargs = {'foo': 'bar'}

print(t1.render(**kwargs))
print(t2.render(**kwargs))

[jamesharris-garmin]

is there anything that can be done about the hard coded reference here? I am not sure I understand how root_render_func is generated to begin with.

[davidism]

I'm not sure there was ever an expectation that templates could be deep copied. As you've seen, templates are specific to the environment that produced them. What's your use case for this?

[jamesharris-garmin]

This is a particularly weird use case within the jenkins-job-buidler tool: https://docs.openstack.org/infra/jenkins-job-builder/ I already submitted a workaround to that tool to late late initialize the jinja template: https://opendev.org/jjb/jenkins-job-builder/commit/b27399c477e5d5331c59aa341b734f5901a7fffe but this would explain our existing use case.