Is there a maximum number of lists one should subscribe to?

[A_Nonimo]

Hi, I’ve always had this question, but only today I’ve finally built up the courage to just ask: is there a maximum recommended number of lists one can subscribe to in Blokada 5.12? I know in ABP there is a recommendation to avoid subscribing to too many filterlists at the same time, so I was wondering if Blokada has a similar problem? If so, how do I know which lists are better?

ooh, and incidentally, is Blokada 5 capable of adding manual filters yet? by that I mean, I just found out somehow Blokada wasn’t blocking googleads.g.doubleclick.net up until 20 minutes ago, is there a way I could have preventively added something like ||doubleclick.net^ so that any domain from a known threat like doubleclick will get blocked, even if it’s not on any of the lists I subscribe to and even if it’s not on my activity log yet either? if there is a way to manually write my own filters, can I use wildcard characters like in ||r*—sn-bg0*.googlevideo.com^? are there more advanced syntax options like blocking specific types of script from a certain domain, but not other scripts or other kinds of request from the same domain?

[Whistler]

That’s a great question about the maximum or optimum number of host lists to use, since it seems reasonable that more lists used could potentially slow internet access. I’d like to hear recommended answers, too.

As for adding manual filters, I’m still using Blokada 4 because I prefer that interface. You can just add blocked hosts as shown in these example screenshots to prevent ES Explorer from calling home. Hopefully, ver 5 has the same feature - maybe a different way to implement it.

(BTW, an app called Network Connection by Antispy Mobile on Google Play Store used to be a good way to determine what urls to block for apps that you want to restrict. Unfortunately, it won’t work as well starting with Android 10 unless the device is rooted. Maybe there’s an alternative app for that, or one could keep an older device with Android 9 just for checking out url calls using Network Connection.)

[A_Nonimo]

@Whistler
I don’t see that option in v5, but it could be just somewhere else. I wish someone who knew would just answer already, the thread closes automatically in 2 days… is there a way to prevent that? Can I just make a new, identical thread after this one closes to give the staff more time to answer?

How many blocklists do you subscribe to? There are some I’m interested in, but I already have 4 (five if you count extensions as separate lists).

In my other front of inquiry, do you use wildcard characters and modifiers in Blokada 4? Is it just like writing filters in AdBlock Plus, AdGuard and uBlock Origin? Or is there a difference?

[Whistler]

Here are some relevant answers:

Wildcard support in Blokada

What are wildcards? Some blocklists use wirdcards to block all subdomains for a domain. Wildcards look like this: *.example.com The asterisk (*) means that anything can replace it, and hence, in this example, any subdomain of the domain example.com will be blocked. Examples of domains that would be blocked: news.example.com something.else.example.com example.com Please note that the wildcard can only be placed as a prefix (before the domain), and only one wildcard is supported per ent…

How to block or allow a domain on Android

This article describes how to block/allow domains on Android. For iOS check this article. The easiest way to do add domains to Ad blocking / Blocked Hosts or Allowed Host is to search the Host log (which is a submenu of Ad blocking) for the domain you want to add. Tap on it and either tap ‘block’ or ‘allow’. If you want to manually enter a domain in one of those menus be sure to follow this format: example.com You can also add the same domain with a ‘www.’ in the beginning. In prior versions…

Screenshot from my ver. 4 host lists saying at top “Try to keep number of active lists low.” But no max number given.

I’m using just one right now (OSID), although I’ve often used two or three at a time. The issues I’ve had with multiple lists is that sometimes certain websites or other internet resources won’t open and I can’t figure out why. After spinning my wheels for a while, I finally remember to stop Blokada, which solves that access problem. Then I switch host list to see if that will still block ads but fix my specific non-access issue. So that’s why I change host lists. Also, using multiple lists will complicate potential access problems.

I don’t see any way to extend the 30-day closing of a support issue other than reposting. Good luck! Maybe someone else will chime in? (Karol?)

[A_Nonimo]

@Whistler Then I guess it doesn’t have any of the cool features browser adblockers have… too sad, I’d been considering switching to Blokada only, but I guess I won’t be. Unless maybe they implement syntax on v6?

If I may repay your kindness, I think there’s an easier way to fix your pages that doesn’t require any of that (at least, that’s what I do here in Blokada 5). Instead of turning off Blokada in its entirety or whole blocklists, go to the activity log immediately after accessing the page (make sure the broken page is the last page you’ve accessed before going to the activity log). Take a look at the last rejected domains (in red rather than green). See if any one doesn’t look ad-y or track-y or malicious and unblock it (add to the exceptions). Reload the page. If it’s not fixed, reblock the domain you just unblocked (remove from the exceptions) and unblock another. Keep trying until you’ve fixed it. An idea is to Google the blocked domain names and see what exactly they actually do; that helps a lot. Eventually, you’ll find out what’s not loading that needed to be loaded, unblock it, and voilà — page fixed. Remember to make sure they page you want is the last one you’ve loaded before each time you go to the activity log and to only keep on the exceptions what you actually need on the exceptions.

You can also use this method to find ad domains that are not on your blocklist and manually add them. I used to do that a lot back when I only had OISD, as that list is good but far from great. But then the developers added CPBL, my favourite blocklist, to the list of blocklists available, so now I never have this problem anymore. I strongly recommend it, it has everything. If I had to choose just one it would be this. It’s continuously updated to include the entire repertoire of EasyList, EasyPrivacy, Malware Domains, MVPS, Peter Lowe’s Blocklist and URLHaus — all in one, automatically curated to eliminate duplicates, and allegedly complemented by adding hosts that are not on any.

PS: the deadline was 7 days, not 30, but it’s OK, apparently it has been automatically extended. I don’t understand why, but I’m not complaining! As you’ve answered my complementary question, that only leaves the one true main question — what is the maximum safe number of lists one can subscribe to simultaneously?

[Whistler]

Thanks for the tips! I’ll give Combined Privacy Block Lists a go - sounds like the right ticket.

I appreciate the info on comparing the log entries to fine tune what’s blocked. (I did stumble onto another post that also discussed that.) Usually, when I can’t get to a particular web page or other internet resource, I want it right now, so it’s simpler to just turn off Blokada for the moment. Then, I don’t get around to troubleshooting further. Perhaps the log can be used to determine sketchy urls that I was previously using Network Connections to find out (mentioned earlier).

I think a benefit of Blokada over browser extensions is that it affects much more than browsers. For example, it blocks ads on Flipboard. It’s somewhat like using a hosts file in Windows to short circuit domain calls, pointing only to the local machine.

I might have seen “30 days” on the initial screen for community support questions. I wonder if just bumping your question will qualify as a reply for auto extension of 7 days.

[A_Nonimo]

Yes, the reason I installed Blokada in the first place was to remove ads from all apps in my phone, and I never saw another ad on a non-browser app again; some of my apps were utterly impossible to use before, now they’re smooth and user-friendly. This is exactly what Blokada purports to do, and it does so flawlessly, even if your only list is OISD (I only got the guts to start subscribing to more after CPBL was made available, now I have Phishing Army too and I’m eyeing others). But it’s also a more aggressive way to block ads on browsers (if I understand correctly, with a browser extension the request reaches the browser and is blocked, whereas with Blokada it doesn’t even reach the phone at all because Blokada diverts it to an entry port that doesn’t exist), and besides, what’s the point of having two adblockers if just one is enough? That’s why I was hoping Blokada were compatible with syntax, so I could just get rid of the browser extension and use Blokada only. Too bad it isn’t, so I’ll have to keep both.

[bongochong]

Just chiming in to say that it’s really about the size of / amount of entries in the lists being subscribed to, as opposed to the number of lists.

For example:

• AdAway hosts is currently ~350kb (approximately 8,800 entries | extremely lightweight, mostly mobile ad-blocking)
• Dan Pollock’s hosts is currently ~475kb (approximately 15,400 entries | basically lightweight general ad-blocking)
• Energized Spark hosts is currently ~1.2mb (approximately 42,500 entries | approaching medium weight general ad & tracker blocking)
• OISD’s basic hosts is currently ~1.35mb (approximately 71,000 entries | medium weight general ad & tracker blocking)
• My own CPBL hosts is currently ~2.5mb (approximately 100,000 entries | upper end of medium weight general ad, tracker, and malware blocking)
• HBlock hosts is currently ~5.5mb (approximately 200,100 entries | lower end of heavyweight general ad, tracker, and malware blocking)
• Energized Blu hosts is currently ~6mb (approximately 230,000 entries | lower end of heavyweight general ad, tracker, and malware blocking)
• Developer Dan’s ad-blocking hosts is currently ~12mb (approximately 368,000 entries | heavyweight general ad, tracker, and malware blocking)
• Energized Basic hosts is currently ~13mb (approximately 400,500 entries | heavyweight general ad, tracker, and malware blocking)
• Energized Ultimate hosts is currently ~21mb (approximately 992,000 entries | extreme upper end of heavyweight general ad, tracker, and malware blocking)

I find that a large majority of ads and trackers one would encounter in daily browsing and application use can be avoided by using one or two regularly updated lists in the medium weight categories, and for many users, a combination of the lightweight lists would suffice. Occasionally things may fall through the cracks, but it’s seldom enough to not be a real issue, and us list maintainers are always improving coverage.

I notice performance issues start to show up on very low memory devices (like Android GO phones) when using lists that greatly exceed roughly 200,000 entries in Blokada (and similar applications like DNS66), so that’s something to keep in mind if you have a real budget device. If your device has 3gb or more RAM, then you can get away with a good deal more, but shouldn’t just subscribe to everything. It should also be noted that the larger a list gets, the more potential there is for false positives and performance issues. Anyway, the best way to choose lists is according to your needs, and the specs of your particular device. I would be hard pressed to say there’s an exact limit to what Blokada can handle, though personally, the largest list I would use from Blokada’s selection on any device would be HBlock.

TLDR: There are four guidelines to follow. 1. Lower memory devices should ideally be using smaller lists. 2. Higher memory devices can get away with using larger lists. 3. Only subscribe to lists which you see as crucial to improving your experience and safety. 4. Avoid subscribing to more than one heavyweight list at a time.

[Whistler]

@bongochong, thanks for that input, and for giving the sizes of example lists! Is there anyway to quickly see size of lists other than what you just provided? It would be nice if Blokada included approximate number of entries (obviously, that would vary over time) in the selection screen.

[bongochong]

No problem at all! And unfortunately there isn’t, but I think that would make for a killer feature, so should probably be requested. The main screen does show how many domains are being blocked in total though, so that is still helpful.

[A_Nonimo]

Wow, @bongochong himself is answering my question!!! This is a tremendous honour, sir!

My phone is a Samsung Galaxy A51 with theoretically 4GB RAM (543MB reserved, apparently, but that’s still more than 3GB) and right now I have OISD (“Standard (Wildcards)”), Phishing Army (“Standard” + “Extended”), CPBL (“Standard”) and URLhaus; I was thinking of adding precisely hBlock and maybe The Blocklist Project. If I may bother you with some questions,

• Since I already have CPBL, does that mean I don’t need a separate subscription for URLhaus?
• Does Phishing Army Extended include Phishing Army Standard, or should they be used together?
• Are there plans to make CPBL Wildcard Blacklist available on Blokada 5? That would block everything CPBL Standard does, but with a smaller list and greater coverage because wildcards, right?
• If I already have CPBL, should I remove OISD? Or, if I install hBlock, should I remove both? In the AdGuard browser extension I have multiple malware lists so that what one doesn’t block, the other does, and I also use CPBL for ABP/uBO alongside AdGuard Basic, AdGuard Mobile Ads, Peter Lowe’s and AdGuard Tracking (is that overkill?)

[bongochong]

Hahaha thanks @A_Nonimo!

With 4gb of RAM I don’t see any problems combining my list with OISD, and even HBlock. There will be some overlap across all three as well, so the resultant amount of blocked domains from combining them wouldn’t be as large as one might expect.

My list does contain URLHaus entries, yes.

I think your AdGuard setup is A-OK. Not overkill.

I think Phishing Army’s standard and extended lists complement one another, though I could be wrong.

I do plan on asking @Karol about switching to the wildcard version of my list at some point, and also to include my “MiniList” hosts file as an option as well, but I move a little slow on such proposals, as the Blokada team is very active in improving the application, and I want to be 100% sure that any issue reports or pull requests I would make are perfectly detailed and error free, to avoid wasted time.

[A_Nonimo]

@bongochong

There will be some overlap across all three as well, so the resultant amount of blocked domains from combining them wouldn’t be as large as one might expect.

I would expect a lot of overlapping TBH kkkkkk that’s why I prefer combined lists like yours, more protection with less lists and less overlaps — but it also makes me kind of weary of subscribing to multiple combined lists, isn’t it a problem when there are too many overlaps? how does Blokada handle an overlap?

@bongochong

My list does contain URLHaus entries, yes.

That means all URLHaus entries, right? Not just some?

@bongochong

I think your AdGuard setup is A-OK. Not overkill.

Thank you =D

@bongochong

I think Phishing Army’s standard and extended lists complement one another, though I could be wrong.

I just sent an e-mail to @drego85, let’s see if he answers

@bongochong

I do plan on asking @Karol about switching to the wildcard version of my list at some point, and also to include my “MiniList” hosts file as an option as well, but I move a little slow on such proposals, as the Blokada team is very active in improving the application, and I want to be 100% sure that any issue reports or pull requests I would make are perfectly detailed and error free, to avoid wasted time.

Is that hard? Does Blokada 5 intend to implement custom subscriptions? Ooh, and what about custom filters, have those been implemented yet?

[bongochong]

Hey again @A_Nonimo. Overlap is not an issue, no. Blokada de-duplicates everything, so no unnecessary entries are kept in memory. I get great mileage out of my own list, and it’s the only one I use, though that doesn’t negate the value of other lists. For the record, I focus on comprehensively blocking ads and trackers, in addition to malware, scam, and software-related telemetry domains. In regard to URLHaus, my list includes only the active domains, so there’s not complete parity, though it is certainly close, and I pull from a variety of other malware databases & lists to ensure that coverage in this regard is also comprehensive. To answer your question as to the difficulty of including other lists, from my understanding, that’s not tough (though quality control is an obvious factor for the Blokada devs), but implementing custom subscriptions in V5 appears to be more complicated a task.

[A_Nonimo]

Thank you =D

@bongochong

Blokada de-duplicates everything, so no unnecessary entries are kept in memory.

Wow, that’s great to know! Completely assuages my fears — I’m under the impression that other adblockers like ABP and AdGuard don’t do that, so I’m delighted to learn that Blokada does! That’s very smart and truly amazing! (and can’t have been easy for the devs to implement) #Blokada rules!

[bongochong]

AdGuard, ABP and uBO definitely do the same BTW. Removal of redundant data is one of the most important aspects in list maintenance/generation, and the applications which allow one to more easily take advantage of these lists. No need to worry about that as long as you’re using something reputable.

If you’re ever curious to see how it’s done at the command line, take a look at the scripts on my github repo, or the repo for HBlock (both of our repos also demonstrate that there are a variety of different approaches one can take to this type of work). Writing routines to parse and structure text can actually be a fun and zesty enterprise! It’s a great learning exercise for anyone looking to improve their skills with regular expressions as well.

P.S. I also want to note that I’m a fan of Héctor’s work on the HBlock script. Whereas my approach to list generation is intentionally compartmentalized, modular and template-based, Héctor largely relies on a highly structured, single, unified script, which handles the generation of several list formats in a self-contained fashion. There are real advantages to both approaches, and it just shows that there’s no one right way to go about this. Two people can end up with very different solutions to the same problem, and still be correct, which is one of the beautiful things about shell scripting.

[Whistler]

@A_Nonimo, you said that you don’t think V.5 has the feature to block specific hosts, but I believe that when you look at the log and see an entry that was allowed, but you want to blacklist it, it looks to me like that will put that questionable entry in the blocked list. I think that’s the technique you were recommending to me. So the feature may be in Ver. 5

Edit: Let me attempt to clarify my comment - in V4, when I go to the log and “allow” an entry that was blocked, Blokada puts it in the whitelist and it gets added to the “Allowed Hosts” list. You had mentioned that in V5 it would be in the “exceptions” list. You also said that you could manually add exceptions. Perhaps it’s just a change in terminology, V4 to V5? (Conversely, in V4, blocking an entry in the log to put it in the blacklist adds it to the “Blocked Hosts” list.)

Edit 2: I’ve been reluctant to reinstall V5 because my past experience of downgrading caused V4 to start the clock over on the gazillion blocked counter, which I love to see. (Also, note on the Blokada download site that V4 “has more features.” However, I thought I’d be clever and install V5 into the secure folder of my Samsung Note 8. You may be aware that the Samsung secure folder installs separate instances of apps. For example, you can text between two Signal accounts that way (useful for testing anyway).

So, now I understand what you’re dealing with in the “My blocklists” (“Blocked” and “Allowed”). I didn’t see any way to manually add an entry in My Blocklists other than by using the Activity log. The manual entry method is in V4 but not V5. Also, I see that a log entry that has been changed to block or allow is annotated with “modified.” I don’t believe the V4 log shows that.

Interestingly, when Blokada - installed in the secure folder - is activated for ad blocking, it only monitors or blocks activity for apps that are installed and run from the secure folder. Main apps and activity on the device are unaffected.

Whoops! I see that V4 is now completely gone from my Note 8, so not so clever after all. I’ve installed V4 again for the device as a whole: “Ad blocking is active, 1 blocked since 11.06.21” I wonder if the “App Cloner” app will allow V4 and V5 to be installed simultaneously (with only one one ad blocking activated at a time).

Oh, I noticed under the List Status of V4 (in Ad blocking settings) this statement: “Filters counter
Filters are now configured with 101,780 rules. Your device allows max 256 MB of memory for every application, and up to 1.93 million rules can fit into it.
Remember, this is just an estimate and Blokada utilises the most memory when it merges the lists. If you enabled a few of them recently, wait a few minutes for the system to free up memory.” And the number of rules is stated on that settings screen as shown in the screen shot below. The number does change consistent with adding more lists; for example, it goes up from 101,780 to 219,354 rules when I add hBlock to CPBL. You can probably calculate the deduplications by playing around with Host Lists selections and checking this number in List status.

Thanks again, @bongochong for a wealth of great information.