WCF icon indicating copy to clipboard operation
WCF copied to clipboard
verified publisher icon WoltLab

Utilized spiderList.xml is outdated

Open IceQ1337 opened this issue 3 years ago • 0 comments

The spiderList.xml at http://assets.woltlab.com/spiderlist/typhoon/list.xml that is used in the RefreshSearchRobotsCronjob to update the list of search robots is very outdated and does not include entries from pre-5.4. Since the list is not tied to major releases, it should be updated more frequently. For example, the "serpstatbot" in version 1.0 was added in April 2021.

Binturong294

I'd like to see the spiderList to be updated after https://github.com/WoltLab/WCF/pull/4761 has been reviewed and (hopefully) merged :)

IceQ1337 avatar Oct 03 '22 15:10 IceQ1337