AdGuardHome
AdGuardHome copied to clipboard
AdguardTeam
Solution to block worldwide services under particular categories
Product: AdGuardHome is a FOSS and On-premises (you own your data) software then i considered to mention resources from other products to get better idea for further development
Why this feature?
- Definitely that'll be parents/employers most loved feature in this product. It means, you don't have to block steam or tiktok then struggle with mind, what'll my kids use next, they usually find something that parents never have heard for life. (ex: there're lot of tiktok alternatives similar chingari, dubsmash, triller, funimate etc)
- Continuously add new services to 'Block services' page just make it too complicated and when it become large list then you cannot recognize necessary services (or type of services) to block easily
- You can block worldwide services under certain category. it'll provide most effective coverage (ex: block chat apps category then whitelist 'Signal' domains for DNS server users to communicate)
- 'Blocked services' component suitable for dominant internet companies and adguardhome is a multi-platform software, already has millions of users (ex: 10M+ only docker downloads) then adding regional services leads (ex: social media),
- to be a one of source that bring foreigners to regional platforms and expand numerous disappointing troubles.
- to be a advertising/new users generate source to those sites or apps
- Missing something? You can check query log (or with possible enhanceable mode https://github.com/AdguardTeam/AdGuardHome/issues/1164) and report to adguard https://github.com/AdguardTeam/AdGuardHome/issues/1133
After checked 3 cloud based providers then below contain appropriate categories(38) list
Parental Control Enhanced parental control (or adult content filtering) (It include personals & dating, Intimate apparel & Swimwear and other adult or adult mixed contents) It seems adguard family dns doesn't block dating sites and similar.
Entertainment Media Sharing Streaming Media Entertainment
Computers Proxies, VPNs, P2P & anonymizers #101 Online Storage & File sharing Search Engines & Portals Visual search Downloads Text-Audio only Content Servers Torrents
Personal web content & social networking Chats Comics, Humor & Jokes Forums & Newsgroups Hosted personal pages Instant messaging Moderated forums Blogs & Wikis Parked sites Social networking
Games Gaming Game playing & Game media Game-cartoon violence
Illegal Activities/Cheating Tasteless & Offensive Illegal/ Piracy software Weapons
Health & Medical Fashion & Beauty Health & Medicine
News News
Recreation/Hobbies Food & Dinning Hobbies & Recreation Sports
Shopping Shopping
Travel Travel
Misc Real estate & property businesses
Drugs / Alcohol / Tobacco use or abuse Alcohol & Tobacco Illegal drugs, use and educations
Resources
Comodo dome shield provide 80+ categories
Adult/Sexual*
- Nudity*
- Pornography*
- Adult contents
- Intimate apparel & Swimwear
- Personals & Dating*
(Note: some words above aren't suitable to add to product in anyway due moral reasons even for adults and children may use/facilitate this software as well)
Arts & Entertainment
- Media Sharing
- Streaming Media
- Entertainment
Business
- Advocacy-NGO
- Auctions & Classifieds
- Business
- Finance & Investments
- Job search & Career development
- Marketing & Merchandising
- Stock Trading
Computers
- Proxies
- Online Storage
- Search Engines & Portals
- Visual search
- Downloads
- Information security
- Online services
- Software-Hardware
- Advertisements & Popups
- Computing & Technology
- Peer-to-Peer
- Text-Audio only
- Content Server
Cultures/Society
- Arts & Society & Culture
Drugs / Alcohol / Tobacco use or abuse
- Alcohol & Tobacco
Illegal drugs/Alcohol/Tobacco use or abuse education
- Illegal drugs
Education
- Education & Reference
Games
- Gaming
- Game playing & Game media
- Game-cartoon violence
Illegal Activities/Cheating
- Tasteless & Offensive
- Illegal software
Health & Medical
- Fashion & Beauty
- Health & Medicine
News
- News
Recreation/Hobbies
- Weapons
- Food & Dinning
- Hobbies & Recreation
- Motor vehicles
Shopping
- Shopping
Personal web content & social networking
- Mobile communication
- Professional networking
- Public Information
- Chat
- Comics & Humor & Jokes
- Forums & Newsgroups
- Hosted personal pages
- Instant messaging
- Web applications
- Kids sites
- Moderated forums
- Blogs & Wikis
- Parked sites
- Social networking
Sports
- Sports
Travel
- Travel
Misc
- Real estate
- Technical Information
- Political issues
- Religion
- Government & Legal
From NextDNS
- Porn
- Gambling
- Dating
- Piracy
- Social network
From Cleanbrowsing
- Adult & Pornography
- Adult mixed contents
- Ads & Tracking
- Torrents
- P2P & File sharing
- Proxy & VPNs
- Gambling
- Dating
- Search engines
- Weapons
- Malicious
- Gaming
- Social networks
Nice. Some additional ideas
- block a category it then maybe be able to enable a certain service (say signal and block all messaging)
- blocking the tiktok category but enable only tiktok for a certain time of day
- give time access for each user (for example 14 year old has access to 30 minutes of tiktok while the 8 year old has none)
Comment Hidden - Off-topic for this feature request
- block a category it then maybe be able to enable a certain service (say signal and block all messaging)
That's possible already then, you can create a DNS allowlist or custom filtering rules. (or it might be possible to enhance, custom filter list editor (web-interface) #1450 to fetch necessary domains from category automatically when type 'signal' and help user to create the allowlist(whitelist) immediately without having to spend time for check query log)
- blocking the tiktok category but enable only tiktok for a certain time of day
There are two options (FRs) for this,
- Setup schedule for blocked services https://github.com/AdguardTeam/AdGuardHome/issues/951 and need to implement one more feature for it about custom blocked services https://github.com/AdguardTeam/AdGuardHome/issues/1692 otherwise only able to do it with pre-defined services similar tiktok and nextdns already has implemented https://github.com/AdguardTeam/AdGuardHome/issues/951 while ago but it cannot apply for each services.
- Timed blocklists and whitelists https://github.com/AdguardTeam/AdGuardHome/issues/1203
- give time access for each user (for example 14 year old has access to 30 minutes of tiktok while the 8 year old has none)
It require below features
- Timed blocklists and whitelists https://github.com/AdguardTeam/AdGuardHome/issues/1203 (flexible for that requirement)
- Different client filter configurations https://github.com/AdguardTeam/AdGuardHome/issues/435
It's easy to say, necessary feature requests already exists that helps to accomplish your requirements, @jeee but adguard may feels confuse though.....
I am of the viewpoint that relying on toggle buttons to block these things are not ideal and also not even properly FOSS-philosophy, since the use of standard filterlists would be preferable and require less effort to check the contents of.
In particular, going to https://filterlists.com/ and searching shalla list, shows raw-domains lists for half or more of the topics listed in the OP.
When will adguard team add this feature to home? eta? It's difficult for us to check-block manually. we dont know what'll other person or kid search on internet, sometimes people decide to use sites heard from friends, media sources or find alternatives. For dns server manager, it can be hundred and thousands of online games, social apps and other sites.. i have experienced it myself with adguardhome and i'm not satisfied about blocked services feature, already gave up. therefore we are using paid service provide by a local company for my institute and our home
It's not planned on any milestone yet.
The feature request is rather complicated, and the problem is websites' categorization data:
- It's unclear where to get it.
- If this data is local, we'll have to come up with an update procedure (which is rather tiresome)
- Alternatively, we'll have to come up with a web service that will supply this data. Also, not a simple thing to do.
It keeps surprising me quite a bit that A) a significant amount of AdGuard Home's userbase use AGH for workplace censorship instead of adblocking, and B) that the 18 👍's to the OP ignored https://github.com/AdguardTeam/AdGuardHome/issues/2230#issuecomment-723939361.
To block Flash/HTML5 game sites, Pokémon GO, and a small amount of gacha game resources, these lists should at least be a good start:
- https://raw.githubusercontent.com/cbuijs/shallalist/master/hobby/games-online/domains
- https://raw.githubusercontent.com/cbuijs/shallalist/master/hobby/games-misc/domains
- https://raw.githubusercontent.com/DandelionSprout/adfilt/master/Pok%C3%A9monNoGOZone.txt
- (Optional) https://raw.githubusercontent.com/DandelionSprout/adfilt/master/AntiAnthroCombatWaifuList.txt
To block sites where one can gamble with money:
- https://raw.githubusercontent.com/cbuijs/shallalist/master/gamble/domains
- https://raw.githubusercontent.com/blocklistproject/Lists/master/gambling.txt
- https://raw.githubusercontent.com/Sinfonietta/hostfiles/master/gambling-hosts
- If this data is local, we'll have to come up with an update procedure (which is rather tiresome)
This way, users can get updates daily and store in a DB file. comodoshield is slow for us. it shall not ideal unless adguard team run many servers. our local provider work as a middle-man between ISP and customer. they specially do data mining to provide number of services. It's like ADGuard DNS analyze queries history to improve ad-blocking but it's much more less than what that company does. Yeah, it's unclear about a good way to get this data
@DandelionSprout adguard home has censor features. no fever, it does more than what we hope in simple centralized environment and take control over entire network.
@ameshkov It's not planned on any milestone yet. The feature request is rather complicated, and the problem is websites' categorization data: 1. It's unclear where to get it.
See below solutions, maybe you can pick affordable services (one or combine multiple) and some companies interest to provide their services for free or under discounts to foss products but do no integrate them directly, we do not like it then adguard has to do separate task before serve it. These companies have threat and adult content filtering facilities, it's possible to use them for improve adguard default DNS and family DNS
However, I noticed several sites pricing too high then as i said above, it's better to get a service for free or find free services.
- https://zvelo.com/solutions/web-filtering-parental-controls
Products: zveloDB URL database, Cyber threat intelligence, PhishBlockList | Intelligence, Malicious Intelligence Plus Metadata
zveloCAT™ - Real-Time Web Content Classification at the Page, Post, or Article Level - https://www.webroot.com/ie/en/business/threat-intelligence/internet/web-classification-and-reputation-services It has two type of services as below, Full suite of threat intelligence services for partner integration via SDK and APIs Basic URL categorization and reputation data for internal usage via online purchase Test site: https://www.brightcloud.com/tools/url-ip-lookup.php
- https://www.cyren.com/ It has web classification service. Products available as for Service providers and Tech companies Test site: google.com https://www.cyren.com/security-center/url-category-check
- https://url-classification.io - URL Classification API is a web based API. Industry based - parental control, ISP, router application, end point protection, AdTech
- https://website-categorization.whoisxmlapi.com https://main.whoisxmlapi.com/ (additionally it has IP geo, DRS, WHOIS and DNS history APIs)
- https://www.netsweeper.com/products/url-filtering-categorization
- https://www.similarweb.com/corp/developer/ Categories and Adult sites APIs
- https://aws.amazon.com/awis '
- https://www.webshrinker.com/data-feed/ - Offline category database
- https://url-classification.io - has offline database
@Aikatsui ideally, we'd prefer an open-source categorization solution for AdGuard Home integration (if that's possible).
Meantime found that many platforms has this facility since long time then adguard cannot delay to implement easily ;)
Comodo dome shield, Cleanbrowsing, NextDNS, SafeDNS, SmoothWall, FortiGuard, iBoss, Barracuda, Sophos Web Filtering, Titan Content Filtering, OpenDNS, Netnany, FlashStart, DNSFilter, Mimecast Web Security, McAfee Web Gateway, Clouduss web categories, Forcepoint, Codeproof, ScoutDNS, GoGuardian and more Dnssafety Diladele, NxFilter
I noticed another interesting feature has in many products, real time analyze content, construction, context, URLs then categorize unknown domains. it's possible to build as offline as aggressive feature (otherwise cloud based(ML) solution for all AG products)
@Aikatsui ideally, we'd prefer an open-source categorization solution for AdGuard Home integration (if that's possible).
If there something mention about open-source then found these but it is less likely usable for the purpose as see from outside.
- Untangle Monitor (Open source and GPL). The database is part of WebRoot's Brightcloud, they've have no API option for that directly as it is not their database https://www.untangle.com/shop/Web-Monitor/
- Clearos open source filter https://www.clearos.com/marketplace/gateway/Content_Filter_Engine
Other resources (Free)
- Sensei in OpnSense https://www.sunnyvalley.io/sensei/ (free and commercial)
- Shallalist, use by SquidGuard, Dansguardian (possible with IPCop or IPFire). i dont know if that maintain well now. http://www.shallalist.de/categories.html http://www.shallalist.de/cgi-bin/stat.cgi (seems not better) https://dsi.ut-capitole.fr/blacklists/index_en.php http://software.othello.ch/mod_dnsbl/
Please take priority consideration on this feature request @adguardteam Two participants have provided their opinion for shallalist. It's free and offline database. can you guys add it?
https://aws.amazon.com/awis
https://aws.amazon.com/alexa/ Alexa web information service is very cheap. personally i believe they have the best database and you can keep in cache every queried domain category some years before re-update. this is a nice feature for all adguard products (@Aikatsui :star2:) 1,001 to 1,000,000 requests/ month - $0.00045 per request Over 1,000,000 requests/ month - $0.0003 per request
Edit:
- get 1 million domain list according to popularity (rank) eg: from google to below (the problem, not sure if possible)
- check the category (maybe it can get at same time?)
It's better to dicuss with amazon tech support and make clear everything before add funds otherwise $450 will be evaporate within second 😅
Interesting, thank you for the info!
Meantime, I'll see if we can use Shalla List.
Shalla normally has licences that discourage use by companies (https://www.shallalist.de/), and it's not 100% obvious if the Shalla team is aware of the GitHub re-uploads of their lists (https://github.com/cbuijs/shallalist/).
If that was to be sorted out, I see no problems with using at least some of Shalla's lists.
Surprisingly, no response yet.
I called to the phone number available on this page (2 days) http://www.shalla.de/impressum.html but they didn't answer.
I noticed mentioned one list offer under creative common license https://dsi.ut-capitole.fr/blacklists/index_en.php recently The maintainer "fabrice prigent" interest about adguardhome, may be already following this project
@ameshkov It has around 60 categories, update daily and the list maintain since 25yrs. It's possible to download via http, ftp, rsync If it good then i suggest to assign this FR to milestone because upto 20 competitors have this feature already.
Thank you for the suggestion! I am still not ready to assign a milestone, though, I'd only be able to do that after the redesign is completed.
Either categorize them or put the labels like (chatting, video streaming, gaming, betting, porn, shopping, photo sharing etc) this on the services.
