Bug - gf Patterns Shouldn't Run Against Error 404 Endpoints

[quantumburnz]

Issue Summary

There's a comment in tasks.py about removing endpoints with 404 errors: https://github.com/yogeshojha/rengine/blob/27d6ec5827a51fd74e3ab97a5cef38fc7f5d9168/web/reNgine/tasks.py#L1042-L1046

This same logic should be applied to gf patterns since there's no point in evaluating non-existent endpoints.

Any other relevant information. For example, why do you consider this a bug and what did you expect to happen instead? gf pattern files are evaluating all_urls.txt as you can see below, which can contain non-existent endpoints. https://github.com/yogeshojha/rengine/blob/27d6ec5827a51fd74e3ab97a5cef38fc7f5d9168/web/reNgine/tasks.py#L1146

I have confirmed that this issue can be reproduced as described on a latest version/pull of reNgine: yes, I'm using a pull from 30Dec21.

Technical details

Ubuntu 20.04.3 LTS Docker version 20.10.12, build e91ed57

[github-actions[bot]]

👋 Hi @quantumburnz, Issues is only for reporting a bug/feature request. Please read documentation before raising an issue https://rengine.wiki For very limited support, questions, and discussions, please join reNgine Discord channel: https://discord.gg/azv6fzhNCE Please include all the requested and relevant information when opening a bug report. Improper reports will be closed without any response.

[psyray]

Fixed by 2.0 version with @ocervell PR https://github.com/yogeshojha/rengine/pull/742 https://rengine.wiki/changelog/#200