pelican-bootstrap3: UndefinedError: 'gettext' is undefined

[chan-y-park]

Hi,

I tried using this pelican theme but it displays CRITICAL: UndefinedError: 'gettext' is undefined and fails to produce html files. I tried using a different theme and it works without a problem.

I am using pelican 3.6.3.

[chan-y-park]

Sorry, I wrote my post at a wrong place...

[justinmayer]

1. You did not mention which theme you are talking about.
2. Current version of Pelican is 3.7, so I suggest trying that before anything else.

[chan-y-park]

Sorry for the confusion; I thought that there is a separate issue page for each theme. The above error message comes out when I use pelican-bootstrap3.

[justinmayer]

Just merged several commits related to that theme, so I suggest using git-bisect to determine which Git IDs work and which do not.

[chan-y-park]

Thanks for your help, I tried git-bisect between the first commit and the current commit and git says b6ab2ad418b634eda366026855be0574703281d7 is the first bad commit.

[chan-y-park]

Hmm, I am not sure if I used git-bisect in a correct way; I tried checking a few commits by hand and the latest commit that works is be36234f9f7fb4633d7e2eee89833839d7cbf1eb.

By the way, I cannot use pelican 3.7, somehow render_math plug-in does not work with pelican 3.7.

[gravyboat]

I ran into this today as well after swapping over to the pelican-themes repo. I'm running this on the 3.7.0 release (just upgraded today) and am getting the same error. Originally it was UndefinedError: '_' is undefined, which was an issue because I didn't have python-gettext installed. After installing that and adding JINJA_EXTENSIONS = ['jinja2.ext.i18n'] it now notes that gettext is missing instead. Note that with 3.7.0 JINJA_EXTENSIONS is deprecated, so it moved to to JINJA_ENVIRONMENT which I imagine is breaking this with how it interacts and what it is expecting maybe? I tried setting JINJA_ENVIRONMENT = {'extensions': 'jinja2.ext.i18n'} and a few other things but that didn't work at all and threw errors related to the namespace.

[gravyboat]

I rolled the theme repo back to https://github.com/getpelican/pelican-themes/commit/be36234f9f7fb4633d7e2eee89833839d7cbf1eb#diff-0eb6cb930365747af1fe070650593b8e (the commit prior to the translation one) and it works fine.

[gravyboat]

@chan-y-park Try installing python-gettext and see if that fixes the error for you on 3.6.x.

[jranke]

The translation feature was developed using pelican 3.6.3 and it used to work for me. Sorry for the inconvenience. I just upgraded to pelican 3.7.0 and got the error about JINJA-EXTENSIONS being outdated. My next step was to use JINJA_ENVIRONMENT = {'extensions': ['jinja2.ext.i18n']}.

Then I still got an error KeyError: 'JINJA_EXTENSIONS' when using the i18n_subsites plugin. So I disabled this plugin, and now I also get UndefinedError: 'gettext' is undefined. I have python-getttext installed, so somehow it is not being found/loaded.

[jranke]

Obviously gettext needs to be installed into the jinja environment using a call to jinja2.Environment.install_gettext_callables as documented here.

However, I do not know how to access the environment that is created with configuration option JINJA_ENVIRONMENT = {'extensions': ['jinja2.ext.i18n']}.

[gravyboat]

@jranke No worries it's not like you hatched a devious plot to break it on this release! I tried something similar to what you did (setting the ENVIRONMENT) variable and it didn't seem to work. I think if we can determine how the mapping works between the JINJA_ENVIRONMENT setting and the old JINJA_EXTENSIONS we can probably resolve the error, I'm just not familiar enough with the internals to know what that mapping looks like.

Edit: Looks like you had the same thought I did!

[jranke]

@gravyboat Thanks for the heads-up! This translation stuff seems not to be used so much in pelican themes...

[jranke]

For what it's worth, I just downgraded pelican to 3.6.3 (from Debian jessie-backports), went back to the previous configuration using JINJA_EXTENSIONS and pelican-bootstrap3 from current master works with this setting. So I wonder why @chan-y-park had this problem with pelican 3.6.3.

[jranke]

It seems to me that this needs to be taken care of in pelican itself, so gettext (and maybe newgettext) can be installed via the pelican config file.

[justinmayer]

Obviously gettext needs to be installed into the jinja environment using a call to jinja2.Environment.install_gettext_callables as documented here.

However, I do not know how to access the environment that is created with configuration option JINJA_ENVIRONMENT = {'extensions': ['jinja2.ext.i18n']}.

This seems to be the relevant question. Any thoughts, @rjames86?

[rjames86]

@justinmayer how was this handled previously? Looks like this isn't a new problem: https://github.com/pallets/jinja/issues/363

In my quick testing in local, adding this to generators.py fixed the problem, but I don't have enough context on the i18n use cases to know if this covers everything:

# On line 71 in generators.py
import gettext
self.env.install_gettext_translations(gettext)

[jranke]

Thanks for having a look. I thought it would need to be install_gettext_callables, as mentioned above? But I have not done any tests and have no context either...

[rjames86]

Thanks for having a look. I thought it would need to be install_gettext_callables, as mentioned above But I have not done any tests and have no context either...

That also very well could be. Does anyone else in this thread use the i18n extension that could verify using either one fixes everything? Pelican successfully builds my site when using install_gettext_translations, but I don't know if if solves issues further downstream in the templates.

Just for completeness, I tested this on both 3.6.3 and 3.7 and the error is the same. I'm confident that this is not related to #1999. However, happy to figure out a good fix for this going forward.

[jranke]

Maybe it is just the jinja version that's causing the difference. I can't test right now, but I have a website using i18n, so I hope to be able to test your proposed fix in the next couple of days.

[chan-y-park]

Dear all, thank you very much for your efforts in trying to resolve this issue!

@jranke I doubled-checked with pelican 3.6.3 + current master and I do get the error I mentioned in the first place, then I checked out be36234f9f7fb4633d7e2eee89833839d7cbf1eb and it works fine.

I am running this on a conda environment, so let me post it here just in case there is any dependency I am missing.

List of Python packages

# packages in environment at /home/chan/anaconda2/envs/python2: # _nb_ext_conf 0.3.0 py27_0 anaconda-client 1.5.1 py27_0 backports 1.0 py27_0 backports-abc 0.4 backports.shutil-get-terminal-size 1.0.0 backports_abc 0.4 py27_0 blinker 1.4 bokeh 0.10.0 py27_0 cairo 1.12.18 6 certifi 2016.2.28 click 6.6 click 6.6 py27_0 clyent 1.2.2 py27_0 jupyter_contrib_core 0.3.0 py27_0 conda-forge jupyter_contrib_nbextensions 0.1.0 py27_0 conda-forge jupyter_nbextensions_configurator 0.2.1 py27_2 conda-forge psutil 4.3.0 py27_0 conda-forge configparser 3.5.0 py27_0 cycler 0.10.0 cycler 0.10.0 py27_0 decorator 4.0.10 decorator 4.0.10 py27_0 docutils 0.13.1 entrypoints 0.2.2 py27_0 enum34 1.1.6 py27_0 feedgenerator 1.9 flake8 2.5.1 py27_0 Flask 0.11.1 flask 0.11.1 py27_0 fontconfig 2.11.1 6 freetype 2.5.5 1 funcsigs 1.0.2 py27_0 functools32 3.2.3.2 py27_0 get_terminal_size 1.0.0 py27_0 ghp-import 0.4.1 glib 2.43.0 1 harfbuzz 0.9.39 1 ipykernel 4.2.2 ipykernel 4.5.0 py27_0 ipython 5.0.0 ipython 5.1.0 py27_0 ipython-genutils 0.1.0 ipython_genutils 0.1.0 py27_0 ipywidgets 4.1.1 py27_0 itsdangerous 0.24 itsdangerous 0.24 py27_0 jbig 2.1 0 Jinja2 2.8 jinja2 2.8 py27_1 jpeg 8d 2 jsonschema 2.5.1 py27_0 jupyter 1.0.0 py27_3 jupyter-client 4.3.0 jupyter-client 4.4.0 jupyter-contrib-core 0.3.0 jupyter-core 4.2.0 jupyter-nbextensions-configurator 0.2.1 jupyter_client 4.3.0 py27_0 jupyter_console 5.0.0 py27_0 jupyter_core 4.2.0 py27_0 lcms 1.19 0 libffi 3.2.1 0 libgfortran 3.0.0 1 libpng 1.6.22 0 libsodium 1.0.10 0 libtiff 4.0.6 2 libxml2 2.9.2 0 llvmlite 0.13.0 py27_0 Markdown 2.6.7 MarkupSafe 0.23 markupsafe 0.23 py27_2 matplotlib 1.5.1 np111py27_0 mccabe 0.3.1 py27_0 mistune 0.7.2 py27_0 mkl 11.3.3 0 mpmath 0.19 nb_anacondacloud 1.2.0 py27_0 nb_conda 2.0.0 py27_0 nb_conda_kernels 2.0.0 py27_0 nbconvert 4.2.0 py27_0 nbformat 4.0.1 py27_0 nbpresent 3.0.2 py27_0 networkx 1.11 py27_0 notebook 4.2.1 py27_0 numba 0.28.1 np111py27_0 numpy 1.11.1 numpy 1.11.1 py27_0 openssl 1.0.2h 1 pandas 0.18.1 pandas 0.18.1 np111py27_0 pango 1.39.0 1 path.py 8.2.1 py27_0 pathlib2 2.1.0 pathlib2 2.1.0 py27_0 graphviz 2.38.0 3 pdrops pelican 3.6.3 pep8 1.7.0 py27_0 pexpect 4.0.1 py27_0 pexpect 4.2.0 pickleshare 0.7.3 pickleshare 0.7.3 py27_0 pil 1.1.7 py27_2 pip 8.1.2 py27_0 pip 9.0.1 pixman 0.32.6 0 prompt-toolkit 1.0.3 prompt_toolkit 1.0.3 py27_0 ptyprocess 0.5.1 ptyprocess 0.5.1 py27_0 pycairo 1.10.0 py27_0 pyflakes 1.2.3 py27_0 Pygments 2.1.3 pygments 2.1.3 py27_0 pygraphviz 1.3.1 pyparsing 2.1.4 py27_0 pyparsing 2.1.5 pyqt 4.11.4 py27_4 python 2.7.12 1 python-dateutil 2.5.3 python-dateutil 2.5.3 py27_0 python-dateutil 2.6.0 python-gettext 3.0 pytz 2016.10 pytz 2016.6.1 pytz 2016.6.1 py27_0 PyYAML 3.11 pyyaml 3.11 py27_4 pyzmq 15.3.0 pyzmq 15.3.0 py27_0 pyzmq 16.0.0 qt 4.8.7 4 qtconsole 4.2.1 py27_0 readline 6.2 2 requests 2.10.0 requests 2.11.1 py27_0 scipy 0.18.0 np111py27_0 setuptools 26.1.1 py27_0 simplegeneric 0.8.1 simplegeneric 0.8.1 py27_1 singledispatch 3.4.0.3 singledispatch 3.4.0.3 py27_0 sip 4.18 py27_0 six 1.10.0 six 1.10.0 py27_0 smartypants 1.8.6 sqlite 3.13.0 0 ssl_match_hostname 3.4.0.2 py27_1 sympy 1.0 terminado 0.6 py27_0 tk 8.5.18 0 tornado 4.4.1 tornado 4.4.1 py27_0 traitlets 4.2.2 py27_0 traitlets 4.3.1 typogrify 2.0.7 Unidecode 0.4.19 wcwidth 0.1.7 wcwidth 0.1.7 py27_0 Werkzeug 0.11.10 werkzeug 0.11.10 py27_0 wheel 0.29.0 py27_0 xz 5.2.2 0 yaml 0.1.6 0 zeromq 4.1.4 0 zlib 1.2.8 3

[avaris]

Let me help here:

The setting change with 3.7 (JINJA_ENVIRONMENT vs JINJA_EXTENSIONS) will not cause a break as defined here.

The issue is that, this "translation" feature was developed with the assumption that i18n_subsites plugin will be present, because that does the installation for you. This interdependence isn't mentioned anywhere (I'm guessing, while developing @jranke wasn't aware of it either).

The problem arises when you don't have the plugin. It doesn't matter what version (3.6 or 3.7) you use. You'll get the error. And with 3.7, you can't use the plugin right now since it needs to be updated to the new setting.

[justinmayer]

Thanks for assisting with this, @avaris. Much appreciated.

@jranke: I wasn't aware of the introduction of a new dependency when merging your PR. Would you please consider preparing a new pull request that makes your I18N features optional? As you can imagine, not everyone needs that functionality, and it should probably be an opt-in feature.

[rjames86]

Thanks @justinmayer and @avaris. At least that validates my assumptions. My change would have no affect, but I didn't know how the i18n stuff worked.

[jranke]

@avaris and @justinmayer: I see it differently: Removing the setting JINJA_EXTENSIONS from the pelican config broke the i18n_subsites plugin.

[jranke]

@justinmayer Sorry, I tried to communicate this in my last comment before the merge https://github.com/getpelican/pelican-themes/pull/441#issuecomment-266811378

[rjames86]

@jranke please take a look at #1999. You'll see that there is no fundamental difference between what was being done with JINJA_EXTENSIONS vs the new JINJA_ENVIRONMENT. The same kwargs are being passed into the environment as before as you can see here: https://github.com/getpelican/pelican/pull/1999/files#diff-1a5861ddd428e9eae60b911254e077b0

[jranke]

@rjames86 yes, but... settings['JINJA_EXTENSIONS'] is gone, which the i18n_subsites plugin uses: https://github.com/getpelican/pelican-plugins/blob/master/i18n_subsites/i18n_subsites.py#L273

[rjames86]

@jranke so going back to the original suggestion, this doesn't necessitate a change to core Pelican. There needs to be an update to the i18n_subsites plugin to pull extensions from the correct location

generator.settings['JINJA_ENVIRONMENT']['extensions']

See settings.py

[jranke]

@rjames86 agreed - should this be conditional on generator.settings['JINJA_EXTENSIONS'] being present? I am not good at python, could you prepare a PR for the i18n_subsites plugin?