Alex Muller

icon indicating a website link http://alex.mullr.net/

icon company @Financial-Times icon location London, UK icon location @alphagov, @praekeltfoundation, @bbc. Now @Financial-Times.

Results 13 issues of Alex Muller

Worker should check the last time a page was crawled

If the crawler discovers a page it should first verify the last time the page was crawled before adding it to the queue. We don't need to re-crawl pages very...

Crawler errors when crawling subdirectories of JSON output

On GOV.UK we have URLs like this: - https://www.gov.uk/api/content/raib-reports - https://www.gov.uk/api/content/browse - https://www.gov.uk/api/content/browse/justice @alext has commented elsewhere: > This is due to a limitation of the crawler. It ends up...

Add RestrictedExtensions to OWASP output

This is the only key which was missing from this output.