Matthias Endler

See #480, which will be the first step towards making line numbers possible. @untitaker fyi.

👍 Comment about the external-stream hack for future reference: https://github.com/lycheeverse/lychee/pull/480#issuecomment-1027301788

Here is an example for how to wrap reqwest with tower: https://github.com/seanmonstar/reqwest/issues/491

There's also https://github.com/antifuchs/governor. We could consider writing a middleware for tower around that.

Also drafted a rate limit parser yesterday: https://github.com/mre/rate-limit There's a lot missing still (please create issues over there!), but I think we can use it to parse actual rate limits...

Also see https://github.com/james7132/Hourai/blob/4a4b343f237f726ae1d2e2943e0bed58406f2cf6/logger/hourai-feeds/src/reddit/rate_limiter.rs

We read the entire file into memory here and we pass that object around: https://github.com/lycheeverse/lychee/blob/6d56c6b55cb01f0c0285419f3ccb89d86495f52a/lychee-lib/src/types/input.rs#L51 https://github.com/lycheeverse/lychee/blob/6d56c6b55cb01f0c0285419f3ccb89d86495f52a/lychee-lib/src/types/input.rs#L279-L281 I was thinking of ways to speed that up. Maybe bstr ist not the...

We could test it and run a benchmark. It probably also depends on the platform. Ripgrep uses mmap, but for some reason not on macOS. https://github.com/BurntSushi/ripgrep/blob/master/crates/searcher/src/searcher/mmap.rs There are some [general...

Maybe all it takes is to remove the `?` in https://github.com/lycheeverse/lychee/blob/c4e004bdf8ee19c4e5945628255851d5498dd400/lychee-lib/src/helpers/request.rs#L52-L53 and replace the match with an `Ok(Some(...))`?

The slash at the beginning might confuse it. That's not a Windows root.