AdGuardHome icon indicating copy to clipboard operation
AdGuardHome copied to clipboard

Exclude certain domains / IPs from statistics

Open adrianmace opened this issue 4 years ago • 22 comments

Prerequisites

Please answer the following questions for yourself before submitting an issue. YOU MAY DELETE THE PREREQUISITES SECTION.

  • [x] I am running the latest version
  • [x] I checked the documentation and found no answer
  • [x] I checked to make sure that this issue has not already been filed

Problem Description

A number of devices - monitoring and updaters - check frequently for the same hostnames. Due to the fact that the TTL is overriden by Adguardhome, these checks greatly skew the statistics. My monitoring device makes 21,600 requests to google.com a day - one every 4 seconds - for example.

My Swann cameras have a bug that's causing a lookup every 5 seconds to www.amazonaws.com. 17,280 times per 24 hours.

Proposed Solution

Offer a way to remove certain domains (www.amazonaws.com) and certain IPs (192.168.1.2) from the statistics gathering process. Still apply the same customised filters and what not, but remove them from the statistics.

Alternatives Considered

Currently bypassing the Adguardhome for these devices affected and going straight to upstream DNS.

Additional Information

PiHole has this, but I like Adguardhome's UI and static Go binary more.

adrianmace avatar May 22 '20 07:05 adrianmace

Well, let's see how many upvotes it gets.

ameshkov avatar May 22 '20 11:05 ameshkov

Well, let's see how many upvotes it gets.

Thanks for getting back to me. All the best.

adrianmace avatar May 22 '20 11:05 adrianmace

One more similar feature request: https://github.com/AdguardTeam/AdGuardHome/issues/1714

ameshkov avatar May 22 '20 11:05 ameshkov

Suggestion: implement this feature on single control panel https://github.com/AdguardTeam/AdGuardHome/issues/481#issuecomment-515673586 in query log then user can apply complex/flexible rules, filters, sorts (date range, time range, hide unwanted domains https://github.com/AdguardTeam/AdGuardHome/issues/594#issuecomment-469129412, hide clients, sort for certain devices) etc and then generate result for web-app wide including dashboard, lists pages (see -->#580), client page (see --> https://github.com/AdguardTeam/AdGuardHome/issues/1336) and keep till user edit or clear it manually same as github.

If add control panel(s) to dashboard then it takes unwanted space but it'll be better if dashboard results (total queries, malware/phishing, adult queries, processing time, top clients, domains top blocked clients, domains etc) able to manage through query log control panel as mentioned above

Similar feature requests #1714 > #594 https://github.com/AdguardTeam/AdGuardHome/issues/594#issuecomment-469129412

Optionally, you can get benefit from #1164 (Collapse feature)

Aikatsui avatar May 22 '20 14:05 Aikatsui

Some images to highlight the problem:
image image

adrianmace avatar May 24 '20 06:05 adrianmace

I wonder would it be enough if we simply add a Client setting: "Do not log queries"?

ameshkov avatar May 24 '20 12:05 ameshkov

That would work for me. The reason they’re named is because I set that up in my search for this setting before realising it didn’t exist.

I’d like to eventually be able to exclude domains, but per-client is good enough if it’s easier / faster to develop.

adrianmace avatar May 24 '20 12:05 adrianmace

I wonder would it be enough if we simply add a Client setting: "Do not log queries"?

If implement, Would it be possible to mod it bit: "Do not display queries"? so we still have them to check when necessary and again user has chance to enable/disable that "do not display" option through client settings.

Aikatsui avatar May 24 '20 16:05 Aikatsui

Is there any info here on when this feature will be ready? I currently have 120k requests in the log coming from my RaspberryPi with Raspberrymatic. Every 15 seconds it checks against my Fritzbox if two devices are online (presence detection). This distorts the statistics enormously :-)

Buddinski88 avatar Jan 11 '22 07:01 Buddinski88

Will only be done after the frontend rework.

Tbh, I'd like to keep this feature purely on the UI side (show smth like "X domains are hidden (N requests)").

ameshkov avatar Jan 11 '22 08:01 ameshkov

Will only be done after the frontend rework. Tbh, I'd like to keep this feature purely on the UI side (show smth like "X domains are hidden (N requests)").

Thank you for the update. In general that seems to be fine. Will that count then also be excluded from the statistics?

Buddinski88 avatar Jan 11 '22 10:01 Buddinski88

In general that seems to be fine. Will that count then also be excluded from the statistics?

No, if it's purely frontend we won't be able to get rid of them there.

ameshkov avatar Jan 11 '22 10:01 ameshkov

In general that seems to be fine. Will that count then also be excluded from the statistics?

No, if it's purely frontend we won't be able to get rid of them there.

First of all, thank you for your feedback and sorry that it took me so long. Basically I understand your answer, but it would only provide a real added value if the numbers are also excluded from the statistics. For example, the statement "5%" requests were blocked would be appropriate. I think this adjustment is not a short term issue, but for the future - I think - everyone would benefit from exculpating certain devices from the statistics. Example from my side. Due to presence detection, my smarthome system asks my Fritzbox every 30 seconds if two devices are logged into the WLAN. That's X thousands of requests in a few days and that distorts the statistics enormously.

Buddinski88 avatar Feb 02 '22 08:02 Buddinski88

Sometimes you might just want to hide some queries from the statistics, while some other times you might not even care of recording some queries at all. I suppose that leaving the user free to choose would provide way better added value in both cases, since IMO the use cases are so vary that all assumptions here could be misleading.

danieletorelli avatar Feb 02 '22 09:02 danieletorelli

I am also +1 for this future request... I have some IOT clients, which spam query log and statistics

MattXcz avatar Mar 30 '22 07:03 MattXcz

+1 from my side. Would really nice to a a filter or exclude option!

Toxicolo avatar Mar 31 '22 08:03 Toxicolo

+1 really an important feature

NemoN avatar Mar 31 '22 08:03 NemoN

+1 !!!!

MSlawik avatar Mar 31 '22 10:03 MSlawik

+1 !!

jamboc88 avatar Mar 31 '22 11:03 jamboc88

Okay, it's too popular to keep it on v1.0.0. We're doing the frontend rework&redesign in v0.108.0, let's squeeze this one there as well.

ameshkov avatar Mar 31 '22 12:03 ameshkov

Stats + query log too.So we dont have these request over and over in log

MattXcz avatar Mar 31 '22 12:03 MattXcz

I also have devices which do HUGE amounts of querys of one or more domains. It would be good to be able to drop specific domains completely. No log at all.

HyperCriSiS avatar Jul 24 '22 01:07 HyperCriSiS

I was going to ask about a slightly different feature - separate statistics out by interface or subnet.

Ultimately I think we're all saying the same thing here - we would very much appreciate a way to filter noise out of statistics. (domain/ips/subnets/interfaces)

danbo avatar Oct 02 '22 01:10 danbo

+1

nemo-ex avatar Mar 10 '23 07:03 nemo-ex

@jiangmw, please use reactions on the OP instead of “+1”-like comments. Thanks.

In case people watching this issue have missed it, the most recent releases of AdGuard Home have the feature of ignoring requests by their query domain names, but it's currently only available through the configuration file. API and UI are coming soon.

ainar-g avatar Mar 10 '23 08:03 ainar-g

Implemented for both domains and clients on Edge and in the upcoming release.

ainar-g avatar Apr 18 '23 13:04 ainar-g

Is it possible to ignore wildcard domains? Adding *.example.com doesn't seem to be working :(

rfgamaral avatar May 15 '23 23:05 rfgamaral

Is it possible to ignore wildcard domains? Adding *.example.com doesn't seem to be working :(

Wildcards doesn't seem to work for me neither on latest v0.107.32. Anyone else?

don-dolarson avatar Jun 30 '23 08:06 don-dolarson

@rfgamaral, @don-dolarson, they aren't supported now. Please upvote #5720. (Do not post “+1” comments as one cannot sort by those in Github; a “:+1:” reaction on the top comment is enough.)

ainar-g avatar Jun 30 '23 08:06 ainar-g