Duplicate links still checked

Open tooomm opened this issue 4 months ago • 1 comments

Running the most recent version (v0.20.0), I still see a lot of duplicates: https://github.com/Cockatrice/cockatrice.github.io/actions/runs/17156704956/job/48675191268#step:3:72

Aug 23 '25 11:08 tooomm

If a link occurs multiple times within one file, this shouldn't happen. Duplicates can still occur if a link appears on two different pages and the link is not in the global cache yet. So there's a race to cache the link between two concurrent tasks. Per-host request handling should fix this. See #1605. Can you try https://github.com/lycheeverse/lychee/pull/1844 and see if the issue still occurs when using that version?

Sep 09 '25 09:09 mre