crate-docs-theme
crate-docs-theme copied to clipboard
crate
[RTD] Backward-compatibility code trips documentation builds on downstream projects
About
Remove RTD backward-compatibility layer/module/code sphinx-build-compatibility again.
- https://about.readthedocs.com/blog/2024/07/addons-by-default/
Why? It only needed to be enabled the other day, because we did not want to go down into yet another rabbit hole at that time.
Problem
The recently added RTD backward-compatibility code in src/crate/theme/vendor/rtd_compat/extension.py::manipulate_config is highly problematic when it comes to TimeoutErrors, as it emits warnings then, which trip the build, because they are treated as errors in our setting.
https://github.com/crate/crate-docs-theme/blob/064424148a1f064b007e96a50976f0fd772dbaba/src/crate/theme/vendor/rtd_compat/extension.py#L14-L131
References
WARNING: An error ocurred when hitting API to fetch active versions. Defaulting to an empty list.
WARNING: An error ocurred when generating the list of downloads. Defaulting to an empty list.
build finished with problems, 2 warnings.
- https://readthedocs.org/projects/crate/builds/26356993/
@amotl is that related to the issue that you were discussing on Slack with @BaurzhanSakhariev?
I don't think it is related to this specific issue. While it might be related to general refurbishments and modernizations at RTD, the other thing is related to networking and connectivity issues, and this one is related to Sphinx, so effectively, totally different domains.
An error ocurred when hitting API to fetch active versions.
Ah, right, both are about rejected requests, possibly by rate limiting machineries. So, well, yeah, in this case both issues are probably connected.
Maybe we need to tell them about the IP address of our Jenkins server?
Hm, not necessarily. In this case, RTD is tripping itself: https://readthedocs.org/projects/crate/builds/26356993/
I've now also observed it happening on my workstation, when working on cratedb-guide in sandbox mode.
Running Sphinx v7.1.2
Running "manipulate_config" from Read the Docs "sphinx_build_compatibility" extension. Consider removing it from your requirements and migrating your documentation accordingly. This extension is useful only as a transition but it will not be maintained.
Warning, treated as error:
An error ocurred when hitting API to fetch active versions. Defaulting to an empty list.
Command exited with exit code: 2
Ah this might explain why we currently don't have the old-version-banner
WARNING: An error ocurred when hitting API to fetch active versions. Defaulting to an empty list.
How should we go about this?
It happens again.
- https://github.com/crate/crate-pdo/actions/runs/14279551151/job/40027377425
- https://github.com/crate/crate-python/actions/runs/14279558610/job/40027394491
- https://github.com/crate/sqlalchemy-cratedb/actions/runs/14279529545/job/40027327340
Running "manipulate_config" from Read the Docs "sphinx_build_compatibility" extension. Consider removing it from your requirements and migrating your documentation accordingly. This extension is useful only as a transition but it will not be maintained.
WARNING: An error ocurred when hitting API to fetch project language. Defaulting to 'en'.
Traceback (most recent call last):
File "/home/runner/work/crate-python/crate-python/docs/.crate-docs/.venv/lib/python3.12/site-packages/requests/models.py", line 974, in json
return complexjson.loads(self.text, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.9/x64/lib/python3.12/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.9/x64/lib/python3.12/json/decoder.py", line 338, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.9/x64/lib/python3.12/json/decoder.py", line 356, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
@msbt: I think we should finally tackle removing this compatibility extension.
I've also reported this to upstream authors.
- https://github.com/readthedocs/sphinx-build-compatibility/issues/3
@amotl how would we go about it? Start fresh and add our things one by one or try to untangle it from the current state?
I think we'd remove the compatibility layer, check what breaks, and fix forward as usual.
- I've staged a patch per GH-595. It will still need ~~more~~ the actual dissecting work.
This would be the most prominent and most effective spot to remove from the compatibility layer, where it currently needs to ask the RTD API for help, while that hasn't been needed before:
- https://github.com/crate/crate-docs-theme/pull/595#discussion_r2031411974
Maybe let's start there, because this is the actual pain point? The other context or environment variable adjustments coming from the compatibility layer, how ever they are interwingulated, are certainly weird from a maintenance standpoint, but do not cause actual pains right now.
It will still need ~~more~~ the actual dissecting work.
I've dissected the spot where requests are made to the RTD API in more detail now. See comments referenced below.
- https://github.com/crate/crate-docs-theme/pull/595#pullrequestreview-2747135403
- GH-596 might be a tailored patch that will add immediate relief without much ado, and without needing to do the full thing.
We hope the syndicate builds no longer trip on CI runs. At least it didn't happen last night. Closing this until further notice.
The problem started happening again more frequently. It breaks linkchecker jobs on CI.
Running "manipulate_config" from Read the Docs "sphinx_build_compatibility" extension. Consider removing it from your requirements and migrating your documentation accordingly. This extension is useful only as a transition but it will not be maintained.
WARNING: An error occurred when hitting API to fetch active versions. Defaulting to an empty list.
Traceback (most recent call last):
File "/home/runner/work/cratedb-guide/cratedb-guide/docs/.crate-docs/.venv/lib/python3.11/site-packages/requests/models.py", line 976, in json
return complexjson.loads(self.text, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
-- https://github.com/crate/cratedb-guide/actions/runs/16964399840/job/48084575017?pr=250#step:4:421
We completed the "ignore those errors" mitigation we intended to add the other day, but failed on selecting the right logging level. Now, it does its job right.
- GH-622
Of course, this doesn't mean the story is complete. While we may close this ticket, because the code no longer trips the build, it still needs to be removed to accompany the most recent upgrade/transition on RTDs end.
- GH-623