Set style parameters at the figure level

[TheChymera]

As discussed at the amazing PythonTeX sprint in Austin, we want to support:

• specifying analysis code in one and only one place, making it usable in all publication formats.
• setting plotting styles at the document level.
• setting select parameters at the figure level (height and width).

As of commit 8e2f8c77 reproduce_me supports style specification per-document (including figure size). Try it out!

However, given that figure size can also be set via matplotlibrc, I think we may want to revise our proposed one-liner solution:

\pyfig[figwidth=4.2, figheight=2.1]{3dplot}

I think it would help reduce the PythonTeX code base and developer workload - and create fewer conventions for users to accustom themselves with, if we handled all of our styling needs via matplotlibrc. Ideally one could apply a matplotlibrc.conf file per-document, and an additional one per-figure (to be loaded after - and potentially override - the document defaults). This would also allow users to customize more things at the figure-level than just width and size (we discussed how this might be both uncommon and unwise, but as @untzag pointed out, for specific cases - e.g. 3d matplotlib - imperfections in an external plotting library might make this a hard requirement).

Pherhaps to prevent too many files from accumulating, we could also provide inline matplotlibrc syntax, e.g.:

\pyfig[figure.figsize : 3.3, 3.1 | axes.facecolor: white]{3dplot.py}

But for the time being, just being able to specify one matplotlibrc file per figure, in addition to one matplotlirc file per document (which is already possible), would allow us to provide flexible styling for fixed analysis code - and work on improving the compactness etc. of the code from there.

[gpoore]

That all sounds good! I've looked through all the code, and here are some comments.

General comments

I like the idea of one matplotlibrc.conf per document type, with a second (optional) conf file providing customization on a per-figure level. Here's the relevant part of the matplotlib documentation.

Currently, you're using figure = latex_figure(...) to generate the figure environment text, and then accessing it with something like \py[3dplot]{figure}. To get per-figure conf, you could instead use a function, perhaps something like this (not tested):

def make_fig(conf=None, caption='', label=''):
    if conf is None:
        fig = latex_figure(save_fig(), caption=caption, label=label)
    else:
        with plt.style.context([os.path.join('pythontex', conf)]):
            fig = latex_figure(save_fig(), caption=caption, label=label)
    return fig

Then, in the .tex file, you would use something like \py[3dplot]{make_fig(conf='3dplot.conf', label='3dplot', caption='A 3D plot')}. This would move the label and caption out of the Python code into the LaTeX document, but there should be a way to define functions differently to get those back into Python if you want. This is just the simplest example I could think of. The main point is that, however you want to do it, with plt.style.context() should give you the kind of per-figure conf that you're interested in.

Shared directory

Currently, all the documents are in a single directory. It looks like this means that the plots (.pgf files) are overwritten by whichever document was compiled last, since they all have the same file names. It would probably be good to give plots names based on document type, put the documents in different directories, or otherwise rearrange things to prevent overwriting.

pres.tex

This uses \input{pythontex/functions.tex} twice. The second one can be removed.

draft.sty and beamerposter.sty

Are these included in case they aren't in the user's LaTeX distribution? TeX Live contains IEEEtran and beamerposter, and they should be available in MiKTeX as well.

functions.tex

• What you're doing to get the path (this_path) works. If you ever need an alternative, you could use pytex.docdir to get the directory of the document, and then go from there.

• from pylab import gcf might be more direct as from matplotlib.pyplot import gcf (pylab involves lots of non-matplotlib things).

• As long as you aren't modifying pytex (normal usage), you shouldn't ever need global pytex.

poster.tex

I'm getting a compile error, but haven't tracked it down yet.

[TheChymera]

Thanks for the positive reception and all the suggestions!

Per-Figure Configuration

Setting custom styles as you suggested failed because the style has to be active while the figure is drawn, whereas the suggested figure saving function only turns it on while the figure is saved. I was able to correctly apply the style (and make PythonTeX use per-figure scripts - yay! - in this commit ). This is done via the new pytex_fig() function which is invoked without a \py[*] container. I actually quite like it. It would be cool if we could include it (or some improved version) alongside with some of the other boilerplate code into PythonTeX proper.

I have also updated the organization a bit, to allow better separation of styles and per-figure-style specification. These specifications (unless expressed relatively to the document base style - which may result in an intransparent mess) would actually be specific to combinations of figure and document type. One issue I have encountered here. is that I get the following error if I try to add the entire RepSeP/poster dir to the PythonTeX deps:

Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.4/pythontex.py", line 62, in <module>
    pythontex.main()
  File "/usr/lib64/python3.4/site-packages/pythontex3.py", line 2676, in main
    do_multiprocessing(data, temp_data, old_data, engine_dict)
  File "/usr/lib64/python3.4/site-packages/pythontex3.py", line 1413, in do_multiprocessing
    if val[0] > start_time:
TypeError: unorderable types: NoneType() > float()

Shared Figure Directory

Certainly an issue. I would suggest having PythonTeX create a new directory on the fly to store the files (it already creates pythontex-files-poster). Ideally this directory might also be hidden, since casual users wouldn't really care to see it (so that would make it e.g. .pythontex-files-{documentname}). Do you think you could implement this?

The reason I think it's very important to keep all the .tex files in the same directory is that we want to encourage people to keep as many things as feasible common between publication methods. This is of course effort-saving (since things don't have to be tracked and updated separately), but habits based on less awesome document management may prompt people to intuitively duplicate stuff if they are handed a directory for each document.

I have also decided to create a separate directory for all the figure scripts analogous to how you might have RepSeP/img/ for images.

draft.sty and beamerposter.sty

I haven't really made up my mind about this. I contribute to the beamerposter package, but there's a big delay (year/s) in between me pushing to upstream and the theme making its way to texlive users. Ideally we could use some external document class on top of which we can easily apply some nice and distinctive defaults from within the document (as is the case for the presentation). Sadly I haven't found any article or poster document class which allows for easy and pretty customization.

There's also the related questions of what and how many non-PythonTeX features should be included in the example documents. More features means easier clone&go, but it also forces defaults on users (which may actually end up impeding clone&go at a certain point). I guess this is more or less the same issue as the theme. I'd really like to know your opinion on this.

[gpoore]

It looks like you've made some good progress on getting things working the way you want!

Per-Figure Configuration

Eventually, I would like to add support for executing external scripts. That might allow streamlining your plotting scripts and pytex_fig() management code. It will be a while till anything like this is possible.

The error you're getting from adding the entire RepSeP/poster directory to dependencies is because the dependency handling is only set to work with files, not directories. Again, another thing that needs to be improved.

Shared Figure Directory

Having a system outside of the code to create directories would require some sort of new interface for specifying these. Otherwise, a figure directory (and potentially other directories) would always be created, whether or not they are used. One option would be to add something like pytex.tempdir, which would be a way to access pythontex-files-poster (or whatever the PythonTeX temp/storage diectory is) programmatically. I would suggest using os.makedirs for now.

I'm generally against hiding directories. It can simplify things, but if you're generating a lot of large files, it can mean that users end up accumulating huge caches that they aren't aware of. I suppose you could also add lots of warnings to users about the existence of the hidden directories.

Styles

It sounds like including custom versions of the style files may be a good approach.

I wouldn't worry too much about the conflict between customization vs. defaults. That seems to be a common issue with LaTeX in general, so it's unlikely that you'll be able to escape it with this project.

Modification to PythonTeX

Some of the things you're encountering, like running external scripts and tracking directories as dependencies, are definitely features that need to be in PythonTeX. As I have time (not often at present, unfortunately), I'm working on getting a new code execution core working with Pandoc Markdown. As soon as I have a very basic version of that running, I'll put it on GitHub so that others can start using it and contributing. Then it will be possible to start adding some of these sorts of features to that system. Even once that system is working, though, it will take some additional work to connect it into PythonTeX and replace the current code execution core.

[TheChymera]

LaTeX Interface

It looks like you've made some good progress on getting things working the way you want!

Thanks ^^ It would be great, though, if we could transfer the parameter passing directly to LaTeX syntax. The current code:

\py{pytex_fig('scripts/3dplot.py', conf='article/3dplot.conf', label='3dplot', caption='A 3D plot.')}

Could be replaced with the more compact:

\pys['scripts/3dplot.py', conf='article/3dplot.conf', label='3dplot', caption='A 3D plot.']

Shared Figure Directory

Otherwise, a figure directory (and potentially other directories) would always be created, whether or not they are used.

I'm unsure I understand this issue with the figure directory. Couldn't it just be generated when a figure actually needs to be saved (and simply re-used after that)? Would this pytex.tempdir be a Python function, or a directory name? I guess the templating should be easy to do to avoid name collisions pythontex-figures-${name_of_document_which_called_the_script}.

I understand your concern with hidden directories, but I'm unsure whether it actually applies to your user base. I'm unsure even if figure accumulation is a real concern. If it is, perhaps a cleanup functionality would be preferable - rather than just relying on the fact that if the folder is shoved into the face of the user he may actually open and clean it up every once in a while?

HTML Output

I'm working on getting a new code execution core working with Pandoc Markdown.

I guess this is rally the biggest leap forward atm. Would an immediate consequence of your success be that one can use pythontex to generate HTML documents from markdown source? e.g. via pandoc myarticle.md -o myarticle.html? Do you know hoe this could then work with e.g. more Pythonic and styling-aware HTML generators (e.g. Pelican)?

[gpoore]

LaTeX Interface

Since you're just using \py to call a function, it should be possible to write a LaTeX macro that serves as a wrapper. I don't have time to write a version that handles keyword arguments, but here's a really simple example. If you look into LaTeX packages for key-value parsing, you can probably create just about anything you would want.

A basic example is below. The #1 would need different handling if it needed to be interpreted as something other than a string, or contained characters that must be escaped in a Python string.

\newcommand{\callfuncwitharg}[1]{\py{func("#1")}}

Shared Figure Directory

Generating a figure directory on the fly only when needed would be nice. I was thinking pytex.tempdir could be a directory name. It would actually be possible to implement it as a property, so that the directory is only created if this variable is accessed.

HTML Output

The goal is to get PythonTeX-style features working with Pandoc Markdown for all Pandoc output formats, including HTML. It may also work with input formats beyond Markdown, though that isn't currently an objective.

This should be easy to use with site generators. Simply use Pandoc to convert from Markdown (with PythonTeX code blocks) to Markdown (with evaluated output), then pass the processed Markdown to the site generator.