Add the ability to run custom code (in any language) with a book build

[choldgraf]

One of the benefits of Sphinx using a "regular" Python file for its configuration (conf.py) is that you can also trigger extra actions along with your build process. For example, we often use this to update files, download data, etc that is then used in the documentation itself.

Since we configure things with a YAML file, this is a little bit trickier because people can't just insert raw Python. However, I think there could be value in making this possible.

I think we could support a configuration option like:

sphinx:
  python: |
    print("python to be executed")

or maybe

sphinx:
  python: |
    path-to-myfile.py

and then this could either be executed before the build finishes, or somehow executed as a part of the build process so that you could do things like inject variables into Sphinx etc for the power users.

What do folks think about this? Any ideas?

[ofajardo]

Super useful. I for example write scripts to collect and cache data from a database that will be used in different notebooks in different ways. It would be cool to trigger that script as part of the build as first thing to do before executing any notebook, and also execute another script at the end to do some cleanup.

[chrisjsewell]

perhaps it would be better, if instead of python code, you specify actual Jupyter notebooks to run. This would allow for a language agnostic way to run these pre/post build steps

[choldgraf]

Ooh that's clever!

[ofajardo]

Maybe it's too difficult, but it would be even better if it could be either a jupyter notebook or if it is not a notebook assume it is an executable. In that way it could be a shell or windows batch script for example. There are things that belong to a script rather than a notebook and I suspect if it is about automating something before or after building a bunch of jupyter notebooks, probably a script is better.

[jbusecke]

I believe I just ran into the need for something like the proposed feature over here

This project is a jupyter book based documentation for our project. I am trying to keep some data (about team members) in a separate yaml file, and then populate a jinja template rst file for our support team page. Along the lines of

Support
=======

.. jinja:: team-data

    {% for person in people %}
    {{person.name}}
    ~~~~~
    {{person.role}}
    
    {% endfor %}

This works when I enter all the data directly in the _config.yaml:

...
sphinx:
  extra_extensions:
    - sphinx_jinja
  config:
    jinja_contexts:
      team-data:
        people:
          - name: Leela
            role: Captain
          - name: Bender
            role: Bender

but when we enter more info for several team members this become quite cluttered IMO.

With a pure sphinx page this can be relatively easily solved by adding something like this

# load data
import yaml
with open('_data/team_data.yml') as file:
    team_data = yaml.safe_load(file)
    jinja_contexts = team_data

which loads all the data from an external '_data/team_data.yml' file. Is there any recommended way to implement this with the current version of jupyter book?

If not I guess this is a big +1 for this feature 😁