Show full URL of retrieved security.txt file instead of just host

[janwillemstegink]

The explanation for security.txt may show a URL that you can copy and paste to avoid typing problems.

See an incomplete path for example in: https://en.internet.nl/site/rijksoverheid.nl/2671942/#control-panel-31

currently: security.txt retrieved from www.ncsc.nl. proposed: (security.txt retrieved from) https://www.ncsc.nl/.well-known/security.txt

[bwbroersma]

An extra enhancement would be to see the complete redirect path, but just showing the full resolved path of the sectxt would already indeed be very useful.

[bwbroersma]

I think I have a patch. Note that is will only update the 'retrieved from', not the 'requested from', since the last one will likely otherwise show /security.txt because the legacy path is tried as a fallback: https://github.com/internetstandards/Internet.nl/blob/d0163daee50e46b199591d2a1932007f84e1e727/checks/tasks/securitytxt.py#L46-L47

---

Some samples of 'Technical details' (of rijksoverheid.nl and example.org):

Web server IP address	Findings
178.22.85.9	security.txt retrieved from https://www.ncsc.nl/.well-known/security.txt.

Web server IP address	Findings
93.184.216.34	security.txt requested from example.org.
...	Error: security.txt could not be located.

---

Another thing is this would break all old reports, since they don't have the url in the database. I fixed this by filling in the url with the host variable for old reports. So the url will only show in new reports (once this is merged and deployed).

Maybe the dot should be removed behind the url for easier copying? It's still there because I just changed the {hostname} to {url}, without changing the message.

[mxsasha]

I think I have a patch. Note that is will only update the 'retrieved from', not the 'requested from', since the last one will likely otherwise show /security.txt because the legacy path is tried as a fallback:

True, though we could preserve the original path. I think this approach is reasonable too.

Another thing is this would break all old reports, since they don't have the url in the database. I fixed this by filling in the url with the host variable for old reports. So the url will only show in new reports (once this is merged and deployed).

Yes, this backfill is a good approach.

Maybe the dot should be removed behind the url for easier copying? It's still there because I just changed the {hostname} to {url}, without changing the message.

We should remove the dot, yes