devdocs
devdocs copied to clipboard
Update Godot docs to include v4.2 and fix older version scraping
This adds docs for Godot v4.2, and does some fixups for scraping older godot releases from the upstream website.
Godot 4.2 was released on Nov 2023: https://godotengine.org/article/godot-4-2-arrives-in-style/
Since v3.5, godot upstream docs have used some new HTML layouts and file structures. Since older versions still need to be scrapable, this introduces another set of versioned filters for godot.
This change also makes small changes to the filters to handle the current website markup for the previous godot doc versions.
The @gdscript and @globalscope entries can't currently be browsed, because of an encoding mismatch between the frontend and backend; I've identified a possible fix in #2200 .
checklist
If you're updating existing documentation to its latest version, please ensure that you have:
- [x] Updated the versions and releases in the scraper file
- [x] Ensured the license is up-to-date
- [x] Ensured the icons and the
SOURCEfile inpublic/icons/your_scraper_name/are up-to-date if the documentation has a custom icon - [x] Ensured
self.linkscontains up-to-date urls ifself.linksis defined - [x] Tested the changes locally to ensure:
- The scraper still works without errors
- The scraped documentation still looks consistent with the rest of DevDocs
- The categorization of entries is still good
QA notes
I generated all of the godot doc versions that the scraper supports, and diffed them against the existing packaged versions to ensure as close of a match as possible.
Command to generate a local copy of the godot 4.2 docs:
bundle exec thor docs:generate [email protected]
Spot checks show that the main differences are:
- the attribution statement, which I updated based on the attribution listed on all of the godot public docs going back to 2.1.
class_@gdscriptandclass_@globalscopefilenames are escaped with%40instead of@. I have no idea how the previous packager got those files to be@symbols, I suspect the godot website used to encode those differently.- minor text edits that have been committed to upstream docs in the meantime
- minor markup changes in the upstream HTML over time (mainly on the 3.x series)
Hi. Thanks for doing this!
I just tried your changes and getting a 403 when trying to download Godot 4.2 docs.
root@990bc51c4450:/devdocs# bundle exec thor docs:download [email protected]
(1/1) Godot 4.2 FAILED (OpenURI::HTTPError: 403 Forbidden)
Done
I tried visiting https://docs.godotengine.org/en/4.2 on a browser and it doesn't look like I've been blacklisted or anything.
Regards
root@990bc51c4450:/devdocs# bundle exec thor docs:download [email protected]
@gordon-boom I haven't done anything around uploading a packaged version to devdocs yet, so I wouldn't expect that to work at this point. If there's a step I need to do as part of this PR to get that working, let me know what it is and I'd be happy to do it!
The command I've been using thus far for testing is:
bundle exec thor docs:generate [email protected]
root@990bc51c4450:/devdocs# bundle exec thor docs:download [email protected]
@gordon-boom I haven't done anything around uploading a packaged version to devdocs yet, so I wouldn't expect that to work at this point. If there's a step I need to do as part of this PR to get that working, let me know what it is and I'd be happy to do it!
The command I've been using thus far for testing is:
bundle exec thor docs:generate [email protected]
Ah silly me. I guess I didn't read the difference between download and generate. I will try that next time.
Just tried it out. Looks good! Let's hope it gets merged.
This works for me.
a7c9db2fe8143675ebcd1d54a93b3c3881c40eea