Smartlist doesn't do cleanup

[m-terlinde]

Actual behaviour

If I hit "clean-up" (actually "Bereinigen" in German), Blokada shows 0 entries for over 10 minutes. If I restart Blokada, the old number of entries is shown. Furthermore, I get the notification, that I don't have any lists configured. To make it work I need to hit "drop filter lists" ("Filterlisten verwerfen" in German). After doing so, it works like a charm for a couple of days.

Expected behaviour

The smart list cleans the entries in a couple of seconds/minutes.

Steps to reproduce

I'm quite not sure, how to safely reproduce this. After the clean-up, it works for a few days.

Configuration

1. Blokada version: 4.7.3
2. Device model: Fairphone 3
3. Android version / ROM: Stock, Android 9

Questions

1. Do you know about recent updates (for example system update, security patch update)? Not at all on OS side. Updated to 4.7 today.

2. How often does this problem happen to you? Every time, I want to clean up my smart list. After cleaning all lists, it's okay for a couple of days.

[ps100000]

Most likely you have a list active that sometimes fails to download. If you try to do a cleanup then Blokada will try to download that list again, which could for example fail because the server can't be reached or because you activated "download on WiFi only" and are using mobile data.

[m-terlinde]

Okay, good to know. There was no error message, that the download failed. The Wifi option is on, but I was connected to my wifi when I tried it yesterday.

Is there any way to see, if the download failed and/or the downloaded list got corrupted?

[ps100000]

I don't think that there is any error message for it as it shouldn't really happen. ^^' You could check Blokada's log for any error messages but don't share the complete log here as it might contain personal info. You can find it at "About Blokada"->"Share log".

[m-terlinde]

I went in flight mode in one of the last nights and now the clean-up is corrupted again.

As you mentioned in #759, I tried to do the "Download lists now" thing, but Blokada isn't able to clean up the SL. I'll try to search the logs later on.

[ps100000]

It would be nice to know what (custom) rules / lists you're using and if you have found any error messages inside the log(if you want to send a excerpt from your log please make sure there is nothing confidential in it). :)

[m-terlinde]

Sorry for the really delayed answer! My lists are:

• StevenBlack Unified
• Energized Basic
• Energized Regional
• Adguard Mobile Trackers & Spyware
• Frogeye First Party Trackers
• EasyList Germany

I have no own blocked hosts. After crawling the logs from this day (yay, I exported them! Lucky me.), I found some syncing errors, see log attached. For me it looks like after the first download error, Blokada doesn't try to download them again, since the exception complains about empty rule sets.

Maybe it isn't a SL bug, but more a fetching bug?

[m-terlinde]

Okay, after doing some more research I got the following:

• at the 4 a.m. sync, the custom list "aHR0cHM6Ly9lYXN5bGlzdC1kb3dubG9hZHMuYWRibG9ja3BsdXMub3JnL2Vhc3lsaXN0Z2VybWFueS50eHQ=" fails for the last two days. It's the EasyList. The string decodes to "https://easylist-downloads.adblockplus.org/easylistgermany.txt", which is definitely the wrong format and not -.- the one linked above. Will fix this.
• I did a "Download lists" command, and the wrong EasyList downloads just fine and says in the logs "1434 entries saved"
• The "Download lists" command works without an error in the log
• After the download, I forced the SL clean-up once again
  • The clean-up runs again into an endless loop
  • The log looks somewhat fine
  • I removed my personal allowed domains from the log file
• I deactivated the EasyList and did a "Download lists" once again
  • No errors in the UI
  • The SL does still an endless loop
  • The log shows the same things as in the example one above, except the EasyList entry

[m-terlinde]

Hey @ps100000, had you any chance to look into this again?