minoru-fediverse-crawler icon indicating copy to clipboard operation
minoru-fediverse-crawler copied to clipboard

Stop checking domains that have been dead for a year

Open Minoru opened this issue 4 years ago • 1 comments

Minoru avatar Nov 06 '21 21:11 Minoru

I'm not so sure about this one anymore. If we stop checking the domain, and it comes back alive, we still won't check it.

We could drastically reduce the rate at which such domains are checked. Currently we do that once a week, which actually isn't that bad?

Alternative design would be to check if the domain is still on somebody's instances list, and if it's not, then we can stop checking it. This relies on everybody else removing the domain from their lists though, which IMHO Is not a good bet.

Minoru avatar Jun 17 '23 09:06 Minoru