Ability to write Jinja loaders that use async/await

[simonw]

trafficstars

I'm writing code within an async framework, using Jinja's excellent async mode.

I want to write a custom template loader that can load templates asynchronously from my database - but as far as I can tell the Jinja loader mechanisms only work with synchronous functions.

Borrowing an example from here I'd love to be able to do something like this:

async def load_template(name):
    row = await db.execute("select body from templates where name = ?", name)
    return row[0]

loader = FunctionLoader(load_template)

I initially implemented a workaround for this by loading templates asynchronously in my own code and compiling them for Jinja with jinja_env.from_string(plugin_template_source) - but then I realized that this doesn't work for templates loaded internally by Jinja due to {% extends ... %} and {% include ... %} blocks.

I'd be happy to help implement this if it's a feature that Jinja maintainers agree should be part of the library.

[davidism]

I'd be fine adding it in, but I have no idea what the API/implementation would look like. Did you have something in mind?

[simonw]

My ideal implementation as a user would be the one listed above - where the FunctionLoader loader can take an async function and magically call it in the right way, by internally using the auto_await() function (assuming Jinja is running in async mode, otherwise it would throw an error): https://github.com/pallets/jinja/blob/1eca37ba6d55c57f837a2126a5e240fd596f2e6d/src/jinja2/asyncsupport.py#L169-L172

The implementation looks tricky though, since I imagine getting {% include %} and {% extends %} working against an async loader would require some changes to the code that the Jinja compiler generates, which I'm not at all familiar with.

It may be more Jinja-like to have a new Loader class called AsyncLoader rather than reusing the existing FunctionLoader though.

[grondman]

I'm using Tortoise ORM all over the place and being able to asynchronously load templates from a DB is something I'd definitely use 🙂

[ssjunior]

I am using a custom loader from database in a Django async application... Is there a way to make the loader async? Otherwise I can't load templates from database.. :(