nbsphinx

nbsphinx copied to clipboard

Dear devs, it would be nice to have some kind of progress indication of the conversion progress (e.g. a log message or even a primitive progress bar).

Thanks for this useful tool!

How are you invoking nbsphinx?

Normally, Sphinx produces quite a few log messages on the terminal, at least one line for each source file (i.e. Jupyter notebook). How does your output look like?

Sphinx also has the option -v, which should increase its verbosity, have you tried that?

Could you please describe in a bit more detail what you have in mind?

I'm invoking with the following command:

LANG=C sphinx-build -b html -d build/doctrees  -j8 -v source build/html 

which produces this output:

loading translations [en]... done
/srv/public/marscher/miniconda3/envs/pyemma_dev/lib/python3.6/site-packages/sphinx/events.py:103: RemovedInSphinx30Warning: autodoc_default_flags is now deprecated. Please use autodoc_default_options instead.
  results.append(callback(self.app, *args))
loading pickled environment... done
[autosummary] generating autosummary for: CHANGELOG.rst, Configuration.rst, DEVELOPMENT.rst, INSTALL.rst, api/coordinates.clustering.rst, api/coordinates.data.rst, api/coordinates.pipelines.rst, api/coordinates.transform.rst, api/generated/pyemma.coordinates.assign_to_centers.rst, api/generated/pyemma.coordinates.cluster_kmeans.rst, ..., tutorial.rst, tutorials/notebooks/00-pentapeptide-showcase.ipynb, tutorials/notebooks/01-data-io-and-featurization.ipynb, tutorials/notebooks/02-dimension-reduction-and-discretization.ipynb, tutorials/notebooks/03-msm-estimation-and-validation.ipynb, tutorials/notebooks/04-msm-analysis.ipynb, tutorials/notebooks/05-pcca-tpt.ipynb, tutorials/notebooks/06-expectations-and-observables.ipynb, tutorials/notebooks/07-hidden-markov-state-models.ipynb, tutorials/notebooks/08-common-problems.ipynb
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 0 source files that are out of date
updating environment: 125 added, 0 changed, 0 removed
reading sources... [  9%] CHANGELOG .. api/generated/pyemma.coordinates.cluster_regspace
reading sources... [ 18%] api/generated/pyemma.coordinates.cluster_uniform_time .. api/generated/pyemma.coordinates.data.FragmentedTrajectoryReader
reading sources... [ 27%] api/generated/pyemma.coordinates.data.MDFeaturizer .. api/generated/pyemma.coordinates.save_traj
reading sources... [ 36%] api/generated/pyemma.coordinates.save_trajs .. api/generated/pyemma.msm.ChapmanKolmogorovValidator
reading sources... [ 45%] api/generated/pyemma.msm.HMSM .. api/generated/pyemma.msm.bayesian_markov_model
reading sources... [ 54%] api/generated/pyemma.msm.estimate_hidden_markov_model .. api/generated/pyemma.plots.plot_flux
reading sources... [ 63%] api/generated/pyemma.plots.plot_free_energy .. api/generated/thermo-api/pyemma.thermo.wham
reading sources... [ 72%] api/generated/thermo-estimators/pyemma.thermo.DTRAM .. api/index_serialization
reading sources... [ 81%] api/index_thermo .. legacy-notebooks/methods/model_selection_validation/model_selection_validation
reading sources... [ 90%] legacy-notebooks/methods/msm_with_given_equilibrium/alanine/alanine .. tutorials/notebooks/03-msm-estimation-and-validation
reading sources... [100%] tutorials/notebooks/04-msm-analysis .. tutorials/notebooks/08-common-problems

waiting for workers...

which produces no output due to -v flag. I just see the "waiting for workers" line for ages until everything has been executed and output files are being written. Probably this is due to the parallel execution? My notebooks take long (~45 minutes) to execute. So I would hook into the the execution processor and have a TQDM progress bar with all cells to be executed (no ETA, but just to see that something is being computed).

This is strange, because Sphinx is supposed to reach 100% when all computations are finished.

Are you using some kind of additional parallelization framework within your notebooks (apart from the Sphinx -j option)?

Could you please try to reproduce the problem in a minimal separate project containing dummy notebooks that only "sleep" for some time to simulate the processing?

This would make it easier to reproduce for me (and others) and hopefully to find the problem.