sphinx-gallery
ENH: Parallel gallery generation
Thinking about this a bit more, I'd expect this not to work for (at least) the matplotlib, mayavi, and pyvista scrapers because these are all global-state based. And then there will be tricky interactions with reset_modules, which also by default does global state stuff at least for matplotlib. So I'm not sure this will ever work at least for the majority of our users :(
I'll copy that over to the issue so more people can tell me how I'm wrong :)
maybe the sphinx folks (@samdoran or @tk0miya) could give hints, or sklearn (@Titan-C @GaelVaroquaux) would be interested in speeding up their builds too ?
It seems this handler is invoked on the bootstrap process of Sphinx (on builder-inited event). So there is no special support from Sphinx framework.
indeed, we're looking at parallelizing the gallery jobs at the extension level