Performance regression with v0.10rc2 sidebar links

[tupui]

While testing the RC for 0.10, I noticed that the Sphinx build time significantly increased.

Here is an example of CI timing out now with RC2 https://github.com/scipy/scipy/pull/16660 On main, we use 0.9 and there is no apparent issue.

[jorisvandenbossche]

Comparing 0.9.0 with 0.10.0rc2 and testing this with the Arrow docs, I also see some slowdown: the total build went from around 9 minutes to around 13.5min (both using the same sphinx version (still 4.2 in my test env), only the theme version differed).

I profiled both cases for the writing files phase (as that phase visually (from the live progress output) seemed to show a slowdown). From a quick look at those profiles, it seems that generate_nav_html is now called more than before. Although I would expect that

https://github.com/pydata/pydata-sphinx-theme/blob/06a35adef9bd23ccd15ca7f31114472e483ef662/src/pydata_sphinx_theme/theme/pydata_sphinx_theme/layout.html#L6-L8

should actually ensure it's only called once.

Profiles

With 0.9.0:

With 0.10.0rc2:

[choldgraf]

I wonder if the problem is because we now generate the navbar in two places. First in the navbar, and a second time in the sidebar (so that the navbar shows in the sidebar at the top on mobile). Maybe it re-calculates the links each time?

Either way, this is almost certainly because of how the navigation links are generated, that's by far the most time we spend. I think that's where I'd look to improve the performance.

[choldgraf]

I think that this issue is potentially a simpler way to resolve this problem:

• https://github.com/pydata/pydata-sphinx-theme/issues/876

[choldgraf]

I looks like the PR I linked above doesn't really speed things up (at least for the SciPy docs).

I am not sure what to do here, I don't have time to debug this issue, can anybody try to find a fix for this? Otherwise we are just going to be stuck in Release Candidate purgatory for a while I guess.

[drammock]

I will try to have a look tomorrow or friday. after that I'm at a project-specific hackathon the following week, so may need to pass the baton.

[choldgraf]

copied over from #878

I was not able to reproduce the slow-down reported in this issue. It seems like main is in-fact faster than 0.9. To conclude this I did the following analysis:

• For 0.9, 0.10rc2, and the PR branch in #878
  • For n_pages in [10, 20, 40, 80, 160]:
    • Add n_pages to the demo site under section1/index.
    • Build the demo site 3 times and take the average time to build

This is what resulted:

This is confusing to me, because it's significantly faster on main compared with the last major release.

So are we really sure that there is a significant slow-down?

[tupui]

Also some findings from #878.

On SciPy the slowdown comes from the use of autosummary templates. Removing these allows to have usual build times. I will open an issue for that.