possibility to create a full host ?

[gallegonovato]

Good afternoon:

First of all apologize if I do something wrong. But it is not my intention.

Secondly thank you for all your wonderful work.

I would like to ask if it would be possible to merge all the hosts : https://www.github.developerdan.com/hosts/ into one for better protection and be able to use it for example in adaway on mobile.

Thanks for your time and help

[lightswitch05]

Hello @gallegonovato - I don't use adaway, so I can't be sure, but does it not allow you to add multiple lists? I don't think there would be much value for me to group my categories into a single list. I don't think the majority of people want to block all my categories: ads, facebook, AMP, dating, etc. I think its better for people to be able to pick and choose.

Surely adaway allows adding multiple lists? Perhaps someone else who uses adaway could help with that.

[gallegonovato]

Good afternoon @lightswitch05

First of all thank you for your time and response.

Adaway does allow you to have multiple sources. In fact that's how I have them. But having them all together in one. It would be much better protection and speed. And adaway sometimes chokes, when you release updates. I already told this to the developer

About if there are people who use all their sources and want to block everything, well I know many people. Because for example they are tired of finding connections from google, facebook and others when looking at a website (for example).

Or even there are people who use lineageos without gaps. Or google stuff. We only use open source software, for more protection.

Because everything is becoming more and more anti-privacy. And more and more I know people who don't have facebook or things like that. But on the webs they put ads and other connections to google and other places.

So if you could unite all would be very useful and helpful on your part.

Thank you very much for all your time and help.

Receive a warm embrace

[dnmTX]

Perhaps someone else who uses adaway could help with that.

No better man for the job but...ping @jawz101 😉

But having them all together in one. It would be much better protection and speed.

Nonsense,there will be absolutely no difference(speed wise).

And adaway sometimes chokes, when you release updates

It sure does,it's not easy for a phone CPU and RAM to process 500,000+ domains,trust me. And you want even more 🙄

[lightswitch05]

I think having the lists separate is actually easier on adaway then a single full list. Each list request comes with an HTTP header Last-Modified, which only gets updated when the list has new entries. I'm not sure if adaway checks that or not, but if it does, then adaway would only need to download the lists that have been updated instead of all of them. Having one single large list would mean having to download the entire thing on each change - which ends up being more to download and process.

Beyond that the extra processing, I strongly discourage my 'aggressive' list and 'hate & junk' list from being used by anyone. They are very personalized to me, and so I do not see those lists being very useful to others. Normally these lists end up causing me more work where someone doesn't like something that is listed on them. Having a single list would mean more people using those lists, and I really do not want that. I've even consider deleting those lists from time-to-time. I don't see any clear advantage of having a single list, but I can think of multiple disadvantages, so I do not intend to take action on this request.

I really appreciate your kind words and enjoy having these discussions. You clearly put a lot of thought and effort into the request, and so I'm sorry to disappoint you with this response. I just do not see it being beneficial to this project and potentially causing a lot more trouble and confusion.

[dnmTX]

Dan @lightswitch05 irrelevant to the subject here but if it's not too much trouble can you compile a list to show how many entries/subdomains(just number count) each of the top level domains cointains. Use that JavaScript magic of yours 😉

[lightswitch05]

@dnmTX I think what you are interested in is already programmed up by our mutual friend Steven: https://github.com/StevenBlack/ghosts

[dnmTX]

Yeah,i know but that would require Python if i'm not mistaking and at the present(kind of old Windows 10 here) don't really want to install more than i should. Other then that,yes,Ghosts is what i need indeed.

[gallegonovato]

Good evening @lightswitch05

It's ok, you don't have to apologize, not at all. That's all there is to it.

I have been using your host files for a long time. I have never had any problems. Even the ones you mention not to use.

Since for example I hate the things you mention in the description of your lists.

Even I do not use Google, Facebook (or anything from this company). No social networks or anything. I use on my mobile lineageos 18.1 without gaps and all from fdroid. Or from github, gitlab, codeberg. And nothing pirated. And less I use Windows for example. So you can imagine... I use fork of what would be mozilla for mobiles, like iceraven, fennec from fdroid or mull from fdroid. But always with user to avoid telemetry and other weird stuff. For example the user I use would be https://git.nixnet.services/Narsil

But for example with your fonts:

https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt https://www.github.developerdan.com/hosts/lists/facebook-extended.txt https://www.github.developerdan.com/hosts/lists/amp-hosts-extended.txt https://www.github.developerdan.com/hosts/lists/dating-services-extended.txt https://www.github.developerdan.com/hosts/lists/hate-and-junk-extended.txt https://www.github.developerdan.com/hosts/lists/tracking-aggressive-extended.txt

(But if you tell me to remove the last two as you mention...).

And combined with:

https://raw.githubusercontent.com/kowith337/CustomHosts-via-hostsmgr/unarranged/hosts.txt and https://raw.githubusercontent.com/kowith337/PersonalFilterListCollection/master/hosts/hosts_leftover_aggressive.txt (this one I have it in ublock and adaway but not using Google stuff as it would be extra). And together with some more as it makes a good very good set. Since even using ublock and blocking there the things of the web and others. Some weird or undesirable connection may escape.

Thanks a million for all your time and help. And Millions of thanks for all your work and effort.

Best regards

[jawz101]

Huge host files are cumbersome. Even the default Adaway list could be trimmed down by 75% and still cover 99% of what you would come across. Maybe a couple thousand entries are ones you actually encounter. If you're into the 50,000+ range you might want to consider a layered approach as well as "trust no one by default."

Ex: for web, use something like NoScript or uBlock Origin in Medium Mode. Even without lists, that will block a majority of domains that come 3rd party. Then you're left with a DNS-based blocklist host or network-wide having to resolve a fraction of addresses it would be doing without. I would rather poke holes to allow a handful of things than try to block everything.

I've been overzealous in the past looking through many many blocklists to add to my setups and ended up thinking this is unsustainable. Companies and domains come and go. Most lists are full of dead domains.

If you are looking to do content filtering (dating, hate groups, warez, games, alcohol) that's another ball of wax.

Otherwise, just use StevenBlack's host file and you should be good.

In short, if you're in the mood to block everything then deny all, only allow what you want, and don't rely on lists from other people.

[lightswitch05]

If pi-hole allowed me to subscribe to a list of wild-carded domains, then it would cut my list down to ~22,000 entries. For what its worth, all the domains on my list have valid DNS responses (which I check regularly and removed dead ones), but there are a lot of wildcard DNS servers out there now, and those are tricky to weed out.

[dnmTX]

Well..pi-hole provides various options for wildcard blocking and at the same time can't do the same from a list? That should be addressed with their team i guess. Logically thinking here the core of the pi-hole is dnsmasq and it's all build up from there and dnsmasq can do that in it's raw version.

[jawz101]

Personally, I switched from checking against active DNS records to comparing against Cisco Umbrella's Top 1 million DNS lookup reports. It's like regardless of whatever entries are active, these 1 million DNS requests from there thousands/millions of customers try to look these up that day. I wish I could have something like a plug-in for pi-hole where "if 3 or more pi-hole instances did a lookup to this domain, log it to a central location (excluding non-public names, edu, military, government, org, and unhelpful domains.)

Or maybe a did-you-see-an-ad/check my logs button to get suggestions from volunteers. I love finding new domains to block but only if they meet my mental checklist.

Blokada on Android has an interesting feature that tries to use a tidy list by starting from the subscribed lists you have set up, logs your blocks for a few days, then makes a tiny list of just what was blocked. Then, each day it compares the previous day's traffic to your subscribed lists again, then adds any new hits to your trimmed list. You end up with a tiny list tailored to you.

[gallegonovato]

Good afternoon @lightswitch05 @jawz101

First of all I apologize for any inconvenience this may cause you.

I wanted to ask you a question if you don't mind.

I happen to have been using lightswitch05 hosts for a while now. But I don't understand why now I'm having problems with Adaway. That's why I wanted to ask you if it happens to you. It turns out that loading them loads. But then when you upgrade, it decompensates others, or themselves. Putting installed a few minutes ago. Or in others that put updated for x time. Or they were already updated, they put installed a few moments ago.

I do not know if it is because the address is https://www instead of https://raw.githubusercontent.com or without the www . Because for example I was making tests with https://sebsauvage.net/hosts/hosts and it does not happen.

I don't know, maybe I messed up and I should open an Issues to the creator of Adaway that is where it causes me the problems. For this I ask for forgiveness.

Thank you very much for your time and help.

[gallegonovato]

Good afternoon @lightswitch05 @jawz101

First of all I beg you not to kill me. Please.....

First because it won't let me edit my previous reply.

And second: In adaway I tried and had installed the links from https://github.com/lightswitch05/hosts#hosts a long time ago but I just took a better look, and searched the repository. And I found https://github.com/lightswitch05/hosts/tree/master/docs/lists . It has in its link the format: https://raw.githubusercontent.com/

Let's see if with this link format Adaway goes. Let's see if I can test it to confirm it.

Sorry for the inconvenience and sorry for my clumsiness.

[jawz101]

I maintain the AdAway host file and you're talking about the adaway Android application.

You should be taking your questions to the adaway xda thread because I'm pretty sure whatever your issue is, it has been asked many times.

https://forum.xda-developers.com/t/app-root-nonroot-official-adaway-v5-8-0.2190753/

[gallegonovato]

Good morning @jawz101

Thank you very much for your reply and help.

I opened an Issues on Adaway's github at the end to report the bug.

But it's a curious thing, since I've been using lightswitch05 hosts, since years. And I never had any problems with them. Until a few weeks ago. But in the end I put the raw links from https://github.com/lightswitch05/hosts/tree/master/docs/lists that have the address https://raw.githubusercontent.com/ and everything went back to normal.

Thank you very much for your help and time. And please excuse me if I did something wrong. It was not my intention.

Thank you very much for your time.

Best regards