Blokada 4: maximum number of filter lists exceeded?

[A_Nonimo]

Hello,

I’ve recently installed Blokada on my dad’s phone and I opted for Blokada 4 over Blokada 5 because I wanted to export my custom block (and allow) rules, and Blokada 5 STILL won’t allow you to add domains to the custom block/allow list manually (as opposed to by selecting them on the “activity” log), or even export your settings so you can carry them to a different device (I’m told “use cloud backup” is supposed to do that, but it doesn’t ask me to specify what cloud I want to backup to, so I’m assuming it would be my Google Drive cloud in my case, my dad’s in his) or synchronise settings between different devices (so domains added to the blacklist in one phone would automatically carry over to the other); I wound up holding the two phones one next to the other and painstakingly adding each domain to the custom block/allow list individually, one at a time, and I must say I now absolutely hate Blokada 4’s interface (Blokada 5’s is much better, can’t there be a Blokada version that has the same features as Blokada 4 but with a Blokada 5-like interface?). This is not what I’m writing this about, though; what I’m writing about is that, when I tried to enable my subscription lists (OISD, CPBL, hBlock, TBLP) on Blokada 4, I got this wierd message that I can’t quite reproduce (as I don’t have dad’s phone with me right now), but which basically said that it couldn’t download all the filters I had selected and that I should select less filters. In my own phone I have the exact same filters enabled and I’ve never seen such a message on Blokada 5. It can’t be a storage problem, as dad’s phone is brand new and Blokada 4 is a lot lighter than Blokada 5. It can’t be a RAM problem either, as dad’s phone is a Samsung Galaxy A52 5G — that’s at least as much RAM memory as my own Samsung Galaxy A51, if not even more RAM memory. So what gives? Does Blokada 4 have a maximum limit of filters much lesser than Blokada 5? If I upgrade my dad to Blokada 5, would I lose all the custom filters I’ve spent a whole hour manually setting up?

Q1. Where did you download Blokada from?
A1. The official channel.

Q2. Which device do you use? Do you use a custom ROM? Do you know which OS version you are using?
A2. Samsung Galaxy A52 5G with Android 11. No root, ergo no custom ROM (at least, that’s how I think it works?); the native ROM is OneUI 3.1.

Q3. Please explain the issue as detailed as possible. Also include patterns you observed (like ‘it only happens on wifi’). The more information the better.
A3. It displayed the message every time I tried to select a new subscription list, or check for updates, or even add a new individual domain to the custom list. I don’t remember the exact message, but I will be posting it ipsīs litterīs (maybe with an accompanying screenshot) as soon as I can.

Q4. Since when are you facing this issue?
A4. Since I started selecting filter lists and adding them to Blokada 4 on my dad’s phone. Blokada 4 was newly installed at the time. I think it came with OISD pre-enabled by default (I know Blokada 5 does); I’m not sure if the message first flashed when I tried subscribing to CPBL or hBlock, but it was definitely already there when I started subscribing to TBLP.

Q5. Did you already try to solve the issue somehow? If so: please state what you did.
A5. No, I didn’t try to solve the issue. I thought it would be a non-issue since Blokada 5 doesn’t have this problem, but now I’ve changed my mind and decided posting the question here.

PS: I’d like to congratulate Blokada on working on all my family’s devices. I’ve recently tried moving to AdGuard because it claims to be able to do so much more than just DNS filtering like Blokada, but I was sorely disappointed — if your device is not rooted, AdGuard doesn’t do anything. Zero. Nada. I guess Blokada is truly the only app that can filter ads, trackers and security threats without root access — and it does so free of charge, and is even an open source project. I recommend that anyone who’s dissatisfied with Blokada get a free 7-day trial of AdGuard AdBlocker — that will teach you to truly appreciate how good Blokada Libre is (or, at least, it sure taught me). And, based on the complaints I’ve seen regarding AdGuard VPN on the AdGuard Forum, I’d say Blokada Premium must also be worlds beyond the competition and a far better destination for your hard-earned money.

[PrintableCharacter]

Hi there,
Unfortunately total RAM is pretty much irrelevant to apps.
Android has an automatic RAM usage cap for blocklists. If this limit is reached this may result in crashes and or the app simply not properly working. Unluckily there’s no way to circumvent this. The RAM limit varies from device to device but is normally a maximum of 256mb per app. Samsung tends to be more restrictive towards this than others. Unluckily there is no way to precisely pinpoint the cap on your device since its up to the system how it allocates its resources hence games get more ram to use as other apps.
V5 isn’t just a reskin hence it tends to perform better and be less battery and ram heavy. RAM caps don’t apply anymore either. Here’s the host counter and ram stats feature request. V5 does not exceed limits anymore. [Feature request] Host counter and RAM size
V5 is a complete rewrite of v4 with very little to no code carried over. Its engine therefore is a lot more futureproof to us and enables us to increase the amount of features over time. The UI offers way more space to add new features as well.
It might be possible to upgrade seamlessly from v4 to v5. If you’ve downloaded v4 from our website and download v5 from our website next they should install next to each other and not overwrite each other.
If you then start v5 your custom rules should be there as well so you can (and definitely should) uninstall v4 before actually starting to use v5

[A_Nonimo]

But is the difference so significant that Blokada 4 can’t handle a total 957744 rules? I’ve checked it on the Settings app, and it says:

Average RAM usage: 444MB
Maximum RAM usage: 479MB
Frequency: 440MB/100%
Z-RAM: 3.8MB/100%

I still don’t understand, so please say it again: if I upgrade my dad to Blokada 5, will he lose the custom rules I manually set up? As in “podosupsurge.com”, “alterupholdcinema.com”, “moedgapers.com”, etc? These rules are not part of a list I subscribed him to, they are from me. User list. Personalised.

Also, I don’t understand how you can claim Blokada 5 has more features:

• In B5 I can’t type a domain and add it to the custom filters manually; I need to actually fall for the domain once and block it via the activity filter.
• In B5 I can’t export or import settings
• In B5 I can’t subscribe to lists not listed in the user interface, nor add a .txt file from my internal storage as a custom list
• In B5 I can’t even see how many rules are active in total
• etc

I always thought it was a consensus that B4 has more features — and is lighter on the internal storage, and compatible with older versions of Android — while Blokada 5 is supposed to be faster and lighter on the battery and RAM usage (allegedly).
I personally like Blokada 5’s interface better, but if it’s supposed to have more features than Blokada 4, I either need the user manual to find them or I need to get a paid licence to access them.

---

EDIT: For the sake of comparison (against Blokada 4 on dad’s Samsung Galaxy A52 5G with Android 11), Blokada 5 in my Samsung Galaxy A51 with Android 10 has

• Average RAM usage: 136MB (indeed a lot less than Blokada 4)
• Maximum RAM usage: 502MB (less than 5% more than B4, so the difference is probably not statistically significant)
• Frequency: 124MB/100% (I guess that’s a lot less than Blokada 4 too?)
• Z-RAM: 11MB/100% (that’s a lot more than B4, right?)

[PrintableCharacter]

@PrintableCharacter

Its engine therefore is a lot more futureproof to us and enables us to increase the amount of features over time. The UI offers way more space to add new features as well.

Again - it will offer more functionality in the future. It’s where blokada is headed not where it is now. One major improvement in v5 is the ability to use blocklists which include wildcards. These block a hell of a lot more content than “normal” lists since they’re smaller in size yet more complex and complete than them. Another improvement is a feature of automatically toggling blokada to use specific settings on specific networks. It can now switch to them automatically.
With the numbers you gave me I’m pretty certain that v4 is on edge on the device we’re talking about. More ram usage might simply not be possible. You could circumvent that by using the smartlist feature though which would require your device to stay on at night and can only provide results after 24 hours.
Just out of curiosity: What are your custom URL sets doing? Why are they necessary to your use case? How did you find them? What are they providing? It’s pretty simple for us to add new lists so if you’re missing a certain list we didn’t come across yet we’re definitely willing to take a look. Blocklist maintainers on the other hand are pretty grateful for every URL contribution made to them so feel free to suggest them new entries as well - some of the maintainers are even active here.
If the entries are manually entered into v4 they should be carried over into v5. To check this you can install both apps simultaneously for a short period of time but do not try to use them simultaneously

[A_Nonimo]

Blokada 5 has been around for some 5 years, hasn’t it? Seems kind of hard to believe such trivial features as manually adding user-blocked domains by typing them would take that long to develop (it’s clearly not a technical difficulty, otherwise B5 wouldn’t be able to add manually add user-blocked domains via the activity log); likewise, I’m told B5 compiles a master list of domain rules from the various lists it subscribes to and that this list is consolidated such as to avoid duplicated entries, so it seems weird that, if the Blokada team was able to accomplish that, they couldn’t add a feature that simply counts the number of entries in that master list (= the number of lines in the corresponding .txt file). If B5 can subscribe to lists via the GUI and make its own, why can’t it subscribe to lists outside the GUI and export/import lists via .txt file? I somehow doubt these features will be added in the near future; if it were so easy to add new features, why would such simple features (in comparison to the much more complex, computationally hard to accomplish features that B5 actually does already have) still be missing after 5 years? For the sake of comparison, over the course of these years they have continued to coexist, while B5 still lacks such features that B4 doesn’t, B4 has successfully incorporated the ability to process lists with wildcards, which is not simple and used to be exclusive to B5. It’s almost as if it’s easier to add a computationally more complicated new feature to B4 than a computationally simpler new feature to B5…

Dad’s device does stay on at night (why would it not? how could he receive calls at night if he turned it off?), but if you tell me updating to B5 won’t reset the custom lists, I think I’ll update it to save RAM and improve performance instead.

My custom entries are mostly ad-serving websites (or sometimes tracker websites with painfully obvious domains such as app.tracku4me.club or c.izitrckr.com) that come up in our routine navigation and aren’t covered by any of the lists we subscribe to. If I see an ad, I use Blokada to isolate the source and block it, which is admittedly a lot easier for me in B5 (though that may be because I joined Blokada in the B5 era, so I never learned to use B4; I know people who actually prefer B4 for the interface rather than for the features). I’ve found that even Yahoo’s ad-server is not blocked by default, which seems hard to believe, but nevertheless true. Though overall, I’d say at least 70% of my custom filters consist of a list of two or three random English words + .com (e.g. pianistrefutationgoose.com, stepmotherillegaldepressing.com); that’s a very common strategy for adservers, and it seems that the lists I subscribe to can’t keep up with it in real time. That said, it’s very likely that most of my custom filters are domains that weren’t on any of the blocklists I subscribe to when I originally added them to my custom list, around the time they first showed up (most of them hadn’t been registered for more than a few days when I blocked them), but have probably been added to the likes of CPBL, hBlock, OISD, etc since; given that Blokada doesn’t give a list of blocked domains per subscription, I have no way of knowing. Perhaps a more energised blocklist would provide a more real-time adaptive protection (pun intended), but the four I have (or probably just the main three) provide me with a relatively complete protection already; it’s very rare that I have to add a new custom filter these days, but the total number of custom lists did build up to several dozen over time. The screenshot attached includes a few likely false positives that I’ve neglected to clean up (such as a crashlytics subdomain and a CDN I can’t even remember why I blocked) as well as a few personal annoyances of mine (e.g. Bank of America) and doesn’t include two hits from Yahoo Mail (Yahoo’s admanager and zillow.com) because I haven’t had a Yahoo account in years, but my parents have accounts and I’ve Blokada’ed it in their devices but can’t add it to my own filter list. Feel free to dig through it if you like, see if you can spot anything worth noticing.

EDIT: image won’t upload on this website, so I put it on Google Drive instead

Screenshot_20220415-105055_Blokada 5.jpg

[PrintableCharacter]

Hi,
V5 was released at the end of 2020.
It’s using DoH dns servers which took quite a bit of work to be implemented, automatic network detection and proper wildcard blocklists. I cannot tell you when the advertised update schedule will be applied back.
In order to report false positives and/or suggest new URLs to be added please contact the list maintainers:
Here’s the oisd form as an example oisd | FP report
GoodbyeAds can be contacted here:
GitHub - jerryn70/GoodbyeAds via website form, telegram or github issue.
Our first and foremost focus is stability, reliability and speed hence a lot of changes are internal and not the addition of new features. The last few months the focus was primarily on iOS since apple implemented a breaking ad framework change. v6 has already been made available on ios as a result of this mess and came with the necessity to now also focus on our VPN and Cloud infrastructure.
An entry counter doesn’t really make sense anymore since wildcard blocklists like oisd block a variety of URLs using only one countable txt entry.

[A_Nonimo]

2020? My mistake then, I could have sworn I saw posts here mentioning Blokada 5 since 2018.

I don’t see why report to the list makers if I can simply add to my personal Blokada when I need to. I’m sure CPBL, OISD, hBlock, etc will spot what needs to be spotted in time without the need for me to report.

I’m not complaining, I’m just pointing out that the developers have other priorities than adding features, and that if it were so easy to add features they would have done the most basic features that are so simple but so unimaginably useful already. I do realise that if the new versions keep coming, but always look identical to the old versions, it must be because the changes are under the hood — bug fixes and stability issues.

PS: A rule counter is very important, because it tells you whether it’s safe to subscribe to a new list, and also whether it’s needed. For instance, I’ve only recently learned that hBlock includes Phishing Army — if there had been a rule counter, I would have noticed subscribing to Phishing Army doesn’t increase the number of entries when I’m already subscribed to hBlock. It’s impossible for me to assess whether being subscribed to TBLP actually changes anything because I can’t compare the number of rules before and after subscribing; if the number of rules TBLP has that OISD, CPBL and hBlock combined don’t is very small, I can simply remove TBLP, but without a rule counter I’ll never know. I also think it’s important to have a grip on how many rules you have in total when considering installing Blokada on a new device with less system resources, so you can optimise — is the combined total small enough that that won’t be a problem? Or is it large enough that you’ll have to get rid of certain subscriptions? How does removing this list change the total rules? What about keeping it and removing that one instead?

[PrintableCharacter]

The issue is that it’s technically impossible to count wildcards properly since they’re blocking a variety of urls with one entry
Here’s a wildcard list

The “*” represents that it’s now totally irrelevant what actually is in front or after the url

Here’s a non wildcard list to compare

As a result the full oisd wildcard list is 6,9 MB in size while the full oisd list without wildcards is 26,8 MB in size.
Here’s the host counter feature request. V5 does not exceed limits anymore. [Feature request] Host counter and RAM size

[A_Nonimo]

I could argue that, as far as a user trying to physically fit Blokada into a phone with limited internal storage and RAM resources is concerned, one entry is one entry — the number of URLs blocked per entry is irrelevant to the size of the resulting .txt file, for whom one entry = one line regardless. That’s why lists with wildcards are useful — they block more URLs using a smaller .txt file, as in the example you literally just gave.

But how about, instead of criticising, I offer a solution? A feature request, for many years from now, when the more pressing features have been added and the developers are open to feature requests from the community. An algorithm that assesses the importance of a specific list to your global Blokada as follows:

1. Select a list, and produce a fictitious Blokada setting consisting of all your current lists minus the one you selected.
2. Scan the list with the fictitious Blokada setting from 1., replacing any wildcards in the list under analysis with a random word or number string; compute how many hosts in this test were not blocked vs how many were as percentages of the total number of hosts tested.
3. Invert 2., i.e. scan the consolidated list resulting from the fictitious Blokada setting from 1., replacing any wildcards therein with a random word or number string, with a new fictitious Blokada setting consisting exclusively of the one list you had selected; compute, again, how many hosts in the experiment were not blocked vs. how many were, as percentages of the total number of hosts tested in this experiment.
4. Compute the absolute size in bytes of the list selected, of the fictitious Blokada setting from 1., and of the actual Blokada setting that consists of the fictitious setting from 1. + the list selected.
5. Compute in percentages the size in bytes of the list selected and of the fictitious Blokada setting from 1. relative to the size in bytes of the actual Blokada setting that consists of the fictitious setting from 1. + the list selected.

This test can be applied for each list individually in order to properly access the contribution of each list individually and how much would be lost by unsubscribing, but the option could also be offered to apply the test selecting more than one list at the same time, so the user could compare combinations of lists and see both how much they weigh and how they perform together vs. separately relative to the full current setup.
Another feature request idea is to allow the user to check for whether a site from their personal block list is now featured in any of the subscription block lists, so the user can tell whether these custom rules can be safely removed without affecting the overall Blokada set up (as the domain would still be blocked without them).

But now this conversation has gone completely off topic, so I’ll just shut up.

[PrintableCharacter]

I’m glad I’ve been able to assist you
Feel free to create new posts these at least actually have content ^^
For more information and updates on the host counter please check where I’ve split the post to.
Enjoy your day

[PrintableCharacter]

@sjhgvr is this Screenshot_20220415-105055_Blokada 5.jpg - Google Drive something for you?

[A_Nonimo]

That’s my custom block list. You asked what was so important about it, out of curiosity, so I just posted it, to satisfy your curiosity. You can read it in a minute or so, see if anything jumps the eyes. I want to keep it because I don’t know which rules have been incorporated into subscription lists and which ones haven’t, and if I toggle them off I don’t think I can toggle them back on afterwards, so I have to assume none have.

BTW, it’s already outdated, added a new Google tag services domain that wasn’t being blocked by Blokada before.