Can't find templates in modularized (JPMS) project

[fwgreen]

I have a modular (JPMS) Maven project that's throwing this exception after adding module-info.java:

com.mitchellbosecke.pebble.error.LoaderException: Could not find template "templates/index.html" (?:?)

There might be need for a fallback in the ClasspathLoader for loading resources from the module path:

//Maybe something like ...
InputStream is = this.rcl.getResourceAsStream(location);
if (is == null) {
    is = this.rcl.getSystemResourceAsStream(location);
}

[zimmi]

I wrote my own loader to access templates inside a specific module. Feel free to use it:

package org.example;

import com.mitchellbosecke.pebble.error.LoaderException;
import com.mitchellbosecke.pebble.loader.Loader;
import com.mitchellbosecke.pebble.utils.PathUtils;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.nio.charset.StandardCharsets;
import java.util.Objects;

/**
 * Loads templates from the given module.
 *
 * @author zimmi
 */
public class ModulepathLoader implements Loader<String> {

    private final Module module;
    private final String prefix;

    public ModulepathLoader(Module module, String prefix) {
        this.module = Objects.requireNonNull(module);
        this.prefix = prefix != null ? prefix : "";
    }

    @Override
    public Reader getReader(String templateName) {
        var path = prefix + templateName;
        InputStream is;
        try {
            is = module.getResourceAsStream(path);
        } catch (IOException ex) {
            throw new LoaderException(ex, "Could not open template \"" + path + "\"");
        }
        if (is == null) {
            throw new LoaderException(null, "Could not find template \"" + path + "\"");
        }
        return new BufferedReader(new InputStreamReader(is, StandardCharsets.UTF_8));
    }

    @Override
    public void setSuffix(String suffix) {
        throw new UnsupportedOperationException();
    }

    @Override
    public void setPrefix(String prefix) {
        throw new UnsupportedOperationException();
    }

    @Override
    public void setCharset(String charset) {
        throw new UnsupportedOperationException();
    }

    @Override
    public String resolveRelativePath(String relativePath, String anchorPath) {
        return PathUtils.resolveRelativePath(relativePath, anchorPath, '/');
    }

    @Override
    public String createCacheKey(String templateName) {
        return templateName;
    }
}

Using it like this:

var module = getClass().getModule();
var templatePath = "/templates/";
var engine = new PebbleEngine.Builder()
            .loader(new ModulepathLoader(module, templatePath))
            .build();

var template = engine.getTemplate("template.html");
// ...

NOTE: The above ModulepathLoader works only when it's in the same module as the templates themselves (the template "package" is always open to the caller in this this case).

Edit: For anyone who wants to tackle this, here are the resource encapsulation rules and also an enlightening clarification by Mark Reinhold.

Brushing up on the rules I see the following paths to solve this:

1. Keep using ClasspathLoader and instruct everyone using modules to a) put all their templates into a directory structure that doesn't form a valid package name (e.g. pebble-templates) or b) add opens my.template.directory; to their module-info.java, thereby opening them unconditionally. No source code changes required.
2. Provide another Loader implementation that allows more fine grained resource encapsulation.

For 2) I see the following possibilities (staying Java 8 compatible):

1. Loader that takes a user supplied Class that's inside the module with the templates (just replace Module by Class in the above ModulepathLoader). That would allow the user to write opens my.template.directory to io.pebbletemplates;instead of opening it unconditionally.
2. Loader that takes a user supplied callback to load a template by name. This could look like this: var loader = new CallbackLoader(getClass()::getResourceAsStream);. This would not require any modification of the user's module-info.java that could get out of sync when moving templates around (the user is the one loading templates from their own module, pebble just uses the result). I'm not sure if method references are executed in the context of the module that created them, I'd have to check, but I'm optimistic because lambdas obviously do.
3. Some trickery with MethodHandles.Lookup that opens the template directory at runtime. Not great, because the code would need to determine if it's running on Java 9+.

I would recommend to do both, i.e. document the rules for using ClasspathLoader with modules and optionally provide the CallbackLoader (or maybe DelegatingLoader?) for more encapsulation, as that seems to be the most flexible option.

[zimmi]

@fwgreen Could you try using the default ClasspathLoader and test both the opens my.template.directory; route and changing the template folder to be an invalid package (e.g. my-templates)? That would help a lot.

[fwgreen]

@zimmi Sorry for the late response. I'm not using Pebble directly but as a component of a web framework called Javalin (I'm just a user, not the maintainer). I merely suggested the change as a similar action was taken by the maintainer of ebean, which also couldn't read resource files in a modularized project.

[zimmi]

It turns out the fix only works for ebean because their configuration file is inside the unnamed (default) package, which is always open. This won't work for pebble I think, because templates are likely spread out over multiple directories. There should be no noticeable difference in using ClassLoader::getSystemResourceAsStream over using the ClassLoader of the pebble classes.

To solve your immediate problem, can you try adding opens templates; to the module-info.java of the module that contains your templates? Another way would be to move the templates to an invalid package directory, like "pebble-templates" instead of "templates". I'll have to think a bit more how to solve this elegantly.

[colesico]

I am using pebble in a multi-module jpms project. To access the templates open the package with templates (opens pkg.name in module-info)