lycheeverse
Cannot suppress redirect output
It seems that some things have changed around redirects recently, and now my output (using the GitHub Action) is littered with redirect information. I cannot figure out how to suppress the redirects; I only want to see the error results.
For example, the output shows something like the following for every redirect:
* [200] <https://some-link-that-was-checked.com> | Redirect: Followed 1 redirect resolving to the final status of: OK.
Redirects: https://some-link-that-was-checked.com --> https://the-link-it-was-redirected-to.com
How do I suppress these? I tried using the --quiet argument x2 with no luck.
It's the ## Redirects per input section that I am trying to suppress.
The # Summary and ## Errors per input sections appear above the redirects, so I have have to scroll up through hundreds of redirect outputs to find the actual errors.
Right now we only print redirects in verbose mode (-v and above). @thomas-zahner, maybe we should introduce a dedicated flag for that?
@mre - I might be doing something wrong then.
I tried adding --quiet to the list of args for the GitHub Action and it didn't suppress the redirect output. I then tried adding the quiet flag a second time in the list of args, and that didn't work either.
Do you have an example of how to suppress the redirects per input info that comes after the summary, when using the GitHub Action?
The default lychee-action args contain --verbose, so maybe that's the problem:
https://github.com/lycheeverse/lychee-action/blob/79de8811d696d79a713d425d265c9870b929fbf2/action.yml#L6
You'd have to overwrite the args.
- name: Link Checker
uses: lycheeverse/lychee-action@v2
with:
# Check all markdown, html and reStructuredText files in repo (same as default, but no `--verbose`)
args: --no-progress './**/*.md' './**/*.html' './**/*.rst'
If that doesn't work, can you show me your lychee workflow? Just paste the relevant parts here.
Yes, the args are overwritten. These are the args I had been using:
- name: Check links
uses: lycheeverse/lychee-action@v2
with:
args: >-
--root-dir "${{ github.workspace }}"
--no-progress
--fallback-extensions mdx
--include-fragments
--include-mail
--exclude-all-private
--exclude-path "(^|\/)\."
--accept 100..=103,200..=299,403,429
"./**/*.mdx"
When I add --quiet to that list, it has no effect. The redirects per input are still printed.
Yes, I tried it locally with a test command that I know does redirect. Works as expected:
echo 'https://www.evernote.com' | lychee --root-dir $(pwd) --no-progress --fallback-extensions mdx --include-fragments --include-mail --exclude-all-private --exclude-path "(^|\/)\." --accept 100..=103,200..=299,403,429 "./**/*.mdx" -
Getting contents for input source: ./**/*.mdx
Getting contents for input source: stdin
Resolving input source: ./**/*.mdx
[WARN] ./**/*.mdx: No files found for this input source
[200] https://www.evernote.com/
🔍 1 Total (in 0s) ✅ 0 OK 🚫 0 Errors 🔀 1 Redirects
Only when I set --verbose do I see the redirects:
echo 'https://www.evernote.com' | lychee --root-dir $(pwd) --no-progress --fallback-extensions mdx --include-fragments --include-mail --exclude-all-private --exclude-path "(^|\/)\." --accept 100..=103,200..=299,403,429 "./**/*.mdx" --verbose -
Getting contents for input source: ./**/*.mdx
Getting contents for input source: stdin
Resolving input source: ./**/*.mdx
[WARN] ./**/*.mdx: No files found for this input source
[200] https://www.evernote.com/ | Redirect: Followed 1 redirect resolving to the final status of: OK. Redirects: https://www.evernote.com/ --> https://evernote.com/
🔍 1 Total (in 0s) ✅ 0 OK 🚫 0 Errors 🔀 1 Redirects
However, lychee-action sets the format to Markdown here.
So you'd have to overwrite that parameter as well. E.g. try --format detailed or setting the format with the foramat arg.
For machine-readable parsing, you can also try --format json.
@mre - thanks for the feedback.
Yes, it was the summary output from the lychee-action that I was talking about. Before the team fixed something with the redirects, the Markdown output was quite nice. It looked good and only showed the issues (errors and timeouts). Then something changed (was fixed) with the redirects and the redirect output is now shown in the Markdown summary, which makes it quite noisy and, if you have a lot of redirects, buries the issues that matter.
I have switched the output format to compact. That only shows the errors, but it's not as neatly displayed as the Markdown was. I guess what I was used to was a compact Markdown output mode.
This is related to https://github.com/lycheeverse/lychee/issues/1312 and the change you talk about was https://github.com/lycheeverse/lychee/issues/1433. I see what you mean. The intention with the new redirect reporting is that users get notified about redirects, because commonly a redirect (at least a permanent redirect) is something users should be made aware of. When lychee reports a permanent redirect, I believe that users should ideally update the URL to the redirected destination URL, because the redirect suggests that the original URL might soon become invalid. So ideally users treat their redirecting URLs just like broken URLs and fix them. (though this is a bit opinionated)
So I would like to propose that we keep the current behaviour of showing redirected URLs in the markdown output by default. But I do understand that users might want a way to disable this. So I think it should be a good solution if we use -q/--quiet to hide the redirects form the markdown output. This way you could still get the "old" markdown output.
Alternatively, we could show a note instead?
lychee detected 23 redirects. For better stability, consider replacing them with final URLs. Use --verbose to list them.
@thomas-zahner - good point about actually fixing the redirects, I hadn't thought of that. 😆
Either solution, showing redirects by default but using the quiet flag to suppress them or hiding them by default and adding a note about using the verbose flag to show them, works for me.
To help users update their links, it would be helpful to know which redirects are permanent or temporary.
For example, this link goes through a mix of redirects:
$ curl -I http://dx.doi.org/10.1007/978-3-540-74061-2_11 -L 2>/dev/null | grep 'HTTP\|ocation'
HTTP/1.1 301 Moved Permanently
Location: https://dx.doi.org/10.1007/978-3-540-74061-2_11
HTTP/2 302
location: http://link.springer.com/10.1007/978-3-540-74061-2_11
HTTP/1.1 301 Moved Permanently
location: https://link.springer.com/chapter/10.1007/978-3-540-74061-2_11
HTTP/2 303
location: https://idp.springer.com/authorize?response_type=cookie&client_id=springerlink&redirect_uri=https%3A%2F%2Flink.springer.com%2Fchapter%2F10.1007%2F978-3-540-74061-2_11
HTTP/2 302
location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fchapter%2F10.1007%2F978-3-540-74061-2_11&code=4baab70f-b3b9-4e47-b51a-9f997705c5cd
HTTP/2 302
location: https://link.springer.com/chapter/10.1007/978-3-540-74061-2_11?error=cookies_not_supported&code=4baab70f-b3b9-4e47-b51a-9f997705c5cd
HTTP/2 200
@katrinafyi, from that list, I wouldn't know which one to recommend to the user. 😕 One could make the argument that https://dx.doi.org/10.1007/978-3-540-74061-2_11 would be the most future-proof, since DOI acts as a proxy in front of the downstream resource and the link uses HTTPS, which is more secure than the initial HTTP link. But I could see an argument for picking one of the Springer links, too. Either way, it might be very hard to automate that selection process.
Not suggesting any automation, just that lychee shows the user the types of redirects that have happened :)
I was thinking it would be a small change to the message. Something like this (very rough idea only):
Followed 1 redirect: http://a.com -[301]-> https://a.com
But maybe this is a different feature suggestion from this thread's original idea :)
I want to know if it is possible to hide redirects, that are just an upgrade from http: to https:. What do you think?
Alternatively, we could show a note instead?
lychee detected 23 redirects. For better stability, consider replacing them with final URLs. Use --verbose to list them.
That's a good alternative. I'm also okay with that approach.
I want to know if it is possible to hide redirects, that are just an upgrade from
http:tohttps:. What do you think?
Currently there isn't. I'm not sure if it makes sense to implement such a distinction into lychee itself. I don't think this is a common use-case. I personally would prefer to not implement it in lychee. You can implement this already yourself by using the redirect map from the JSON output where you filter for the entries where the first URL starts with http and the final URL starts with https.