Add option to reconstruct XML from the pages API

[GERZAC1002]

Merged: Add support to backup pages using API:Query instead of Special:Export https://github.com/WikiTeam/wikiteam/pull/280 into recent version of dumpgenerator.py (so, most credit should go to NyaMisty who tried to merge this back in 2016 and closed the pull request back in 2018)

Adds additional parameter: --apiquery which uses a query request instead of submit request on api.php which works without Special:Export which is disabled on some sites.

Also adds: --ignore-api-check which i just copied from the earlier mentioned pull request and it doesn't do much so it could be removed as well

I stumbled into the problem when trying to download https://pokewiki.de where Special:Export wasn't available and i tried the old version from 2016 with the pull request #280 which seems to work so far. I am still doing some testing as I am mostly just trying to archive game wikis and have not much coding experience. I didn't wanted all revisions and only the current once which didn't work because of the missing 'Special:Export' page. Also '--xmlrevisions' wasn't working either as it failed after some time and resuming the progress only lead to it downloading the same pages over again creating many duplicates inside the output file and not going further.

so some improvements I haven't done would include:

• checks if something will break by merging it as i am not using '--xmlrevisions' as it caused more problems for me than it solved. I dumped a few wikis with the new version with this options:'--namespaces 0,4,6,8,12,14 --xml --image --curonly <url> '
• checking for the availability of 'Special:Export' page before retrieving all the namespaces to save bandwidth of the wiki-owners
• maybe change back some timeouts, I adjusted some for my self as i noticed some slower less performant wikis took longer to respond when dumping everything

Thank you all for the great work :) I am just trying to give something back to the community even if it is just recycling old pull requests that got closed.

[nemobis]

Il 12/04/22 01:54, Gernot Zacharias ha scritto:

I stumbled into the problem when trying to downloadhttps://pokewiki.de where Special:Export wasn't available and i tried the old version from 2016 with the pull request #280 which seems to work so far.

Thank you but I don't understand, have you tried --xmlrevisions which uses the API and should work just the same? Did it fail and how?

[GERZAC1002]

I tried --xmlrevisions which always failed and resuming basically made it write the same pages over and over into the output file at that point it was easier to resolve the other issue using the older pull request:

[nemobis]

Il 12/04/22 11:02, Gernot Zacharias ha scritto:

I tried --xmlrevisions which always failed

Thanks. Please open an issue with the command you used and ideally a tail of the XML file after the failure, if you still have it.