Asynchronously deferred content generation.

[amcgregor]

Note: Unlike #2, this ticket is more about the inversion of the template include pattern and less about the pure integration of cinje into an asyncio environment, though such integration does need to be kept in mind.

Currently the yield mechanics are used to identify an insertion point for additional content in a template, creating a "wrapping" template function. This gives you an entry point of the most specific template, which then typically dives into its wrapper, then deeper into whatever that is wrapped by internally, etc., etc.

The inverse approach, where you have a "layout" template which you populate with content blocks, is also extremely useful. Because templates are first-class functions, you can easily pass individual, or whole lists of template functions around, using functools.partial to bind values to them for use at render-time. This is a start, as during template processing the layout template would descend into each content block to render, blocking the overall process. Instead, when a child template is encountered, if it's an async def it can be sent up to the reactor for processing. In the resulting HTML insert a placeholder or marker (a la the animated bars on Facebook posts as they load), finally streaming the layout template, then individual content blocks, as MIME multipart. Should also optionally be able to execute the template more classically, without async deferral.

Streaming chunks to the browser as completed can be accomplished via MIME multipart AJAX, via something like mpAjax. A rough p-code example using Futures:

def layout(reactor, *blocks):
    """Mock boostrap row/column layout."""

    # Prepare some content.
    _buffer = []
    _tasks = []

    for block in blocks:
        if not _buffer or not block:
            if _buffer:
                _buffer.extend(('</div><div class="row">\n', ))

            _buffer.extend(('<div class="row">\n', ))

            if not block:
                continue

        _tasks.append(reactor.submit(block))
        _buffer.extend(('<div class="placeholder" data-await="', id(_tasks[-1]), '"></div>\n'))

    return (''.join(_buffer), _tasks)


def render_page():
    identifier = "gc0p4Jq0M2Yt08jU534c0p"

    response.content_type = "multipart/mixed; boundary=" + identifier

    page, content = layout(executor, [
            "some render function, first column",
            "some other render function, second column",
            None,
            "lastly a full width footer",
        ])

    yield page

    for chunk in as_completed(content):
        yield '--' + identifier + '\nIdentifier: ' + str(id(chunk)) + '\n\n' + chunk.result()

    yield '--' + identifier + '--\n'