[feature] Anubis mitigation

[jtagcat]

(With ~200 daily/weekly crawl jobs), it's becoming apparent tens of sites are dropping off with Anubis.

Alternatives: Enable JS on CDio instance, but that has the con of the page (diffable) content being way more noisy.

These are examples I can share, and could be solved by making 50 or so accounts on a wide range of sites. However, I have many which don't have accounts, even static sites.

I wish there was a way to get a small number of tokens globally without site's administrator manual configuration. This is a more an unsolved problem for Anubis, than for CDio, but creating issue here, as 20% of sites have already gone offline.

My distribution of sites is:

• 59 sites without RSS, which CDio bridges the gap for.
• 58 tagged updates: when something on the page (often factual statements / specs) change.
• 38 on info about organizations
• 18 for once-in-a-moon application rounds
• ...

[dgtlmoon]

kinda duplicate https://github.com/dgtlmoon/changedetection.io/issues/2198

[dgtlmoon]

I wish there was a way to get a small number of tokens globally without site's administrator manual configuration.

You really did not explain the issue here, what do mean by "tokens" and where do they come from? what exactly are you talking about?

[dgtlmoon]

And what is the goal of this post? theres not enough info here other than "anubis does something"

[jtagcat]

I wish there was a way to get a small number of tokens globally without site's administrator manual configuration.

You really did not explain the issue here, what do mean by "tokens" and where do they come from? what exactly are you talking about?

See https://github.com/TecharoHQ/anubis/issues/326. This issue's goal 2 is a ping for possible collaboration (or more correct, communication between parties).

The first goal is to have Anubis specifically tracked, as its adoption is rapid. I'm expecting at least 50% of the previously working sites to be dropped — a core threat (CDio just works low-effort → it maybe works 20% of the time, why bother).

I understand there isn't an actionable item here, you have all the rights to close and ignore.

[dgtlmoon]

Does it say ANUBIS in the "Stats" ?

[jtagcat]

No, I found nginx on one, and apache on an another.

[Xe]

Hi, lead dev of Anubis here. Anubis tries to hide itself as much as it can and won't expose itself in Server headers for this reason. You can probably test for Anubis being in the mix by looking for the string /.within.website/x/cmd/anubis in responses.