uBlock Origin should have a way to fix serious filter issues as soon as possible

[ghost]

Prerequisites

• I performed a cursory search of the issue tracker to avoid opening a duplicate issue
• [] The issue is not present after wholly disabling uBlock Origin ("uBO") in the browser
• [] I checked the documentation to understand that the issue I report is not a normal behavior

I tried to reproduce the issue when...

• uBO is the only extension
• using a new, unmodified browser profile

Description

I reported a similar issue(#1645) before, but @gorhill declined, citing server load and filter-list maintainer measures.

Declined, as I do not want uBO to be more of a burden to servers I do not own, and I do not want uBO to use a server I own to keep the privacy policy dead simple. Maintainer of EasyList has put in place measures to minimize likelihood of such mistake happening again.

However, a serious breakage occurred again. The bad generic filter ##.ads is valid as filter, so it is difficult for the filter-list author to take its countermeasures.

Also, there have been several recent remarks from security researchers about the possibility of creating malicious filters. One has yet to be resolved. A hotfix filter-list will be a way to mitigate the security issue if the filter-list author's account is hacked.

For these reasons, I propose a revised version of my previous proposal.

---

• Create a new filterlist in the uAssets repository.
• It is usually an empty file. Only in case of serious incidents, the uBO-team will add a minimum number of filters, which will be removed after a few days.

• (new) uBlock Origin will check its changes twice, 6 hours and 24 hours after after updating EasyList.

---

For most users, especially install-and-forget users, the filter-lists should all be updated at the same time. So we can use EasyList update time as the basis.

The Gmail issue was fixed 3 hours after the bad filter addition. The Google Docs issue on April was fixed 6 hours and 19 minutes after the bad filter addition.
Based on these two observations, I estimate that six hours optimal to solve critical issues as soon as possible.
However, if a critical issue is reported when, for example, the entire uBO team is busy, it may not be possible to solve it in 6 hours. In addition, high-impact but non-critical problems are not fixed so quickly. For example, the fix for the Google News breakage took 18 hours.
Therefore, checking the changes again 24 hours after EasyList update will ensure that they are corrected including non-critical issues.

A specific URL where the issue occurs

https://twitter.com/search?q=gmail%20broken&src=typed_query&f=live

Steps to Reproduce

Just search twitter.

Expected behavior

The issue fixed 3 days ago. No longer people suffer from it.

Actual behavior

I can find breakage reports easily.

Configuration

uBlock Origin: 1.41.9b0
Firefox: 97
filterset (summary): 
  network: 79763
  cosmetic: 41303
  scriptlet: 16472
  html: 624
listset (total-discarded, last updated): 
  default: 
    user-filters: 0-0, never
    ublock-filters: 30887-27, now
    ublock-badware: 3903-3, now
    ublock-privacy: 208-0, now
    ublock-abuse: 72-0, now
    ublock-unbreak: 1730-0, now
    easylist: 63996-615, now
    easyprivacy: 26629-571, now
    urlhaus-1: 8437-0, now
    plowe-0: 3673-2, now
filterset (user): [empty]
modifiedUserSettings: [none]
modifiedHiddenSettings: [none]
supportStats: 
  allReadyAfter: 439 ms
  maxAssetCacheWait: 183 ms

[u-RraaLL]

Wouldn't such a list mess with selfies?

[krystian3w]

But cosmetic filters no support ,badfilter to the best way as disable typo.

[u-RraaLL]

I think the proposal was more with tech illiterate people in mind.

But yeah, there's more drawbacks than merit to the idea.

[Yuki2718]

[Update] !#include doesn't actually save the connection.

uBO doesn't distinguish each list and setting EL as a clock is analogous to the idea of priviledged list discussed in another issue. This can't be done by trivial code change, what if user disabled EL? Also I don't stand by the idea of adding a list only to fix very rare serious breakage, but it's worth consideration to add a list for whatever hot fixes to fight against serious circumventor such as T.P.U. Frankly, without shorter update interval our effort don't fruit much¹ - see how many updates were done by @JobcenterTycoon in last few days. We can also use the list for serious breakage. But adding a list with update frequency of 1 day means 1.8 times larger number of connection than the current one², and I guess @gorhill is not willing to simply add such a list.

Here's my point: there's no need to set update frequency of all the built-in lists uniformly - Privacy and Resource abuse are sparsely updated and not getting the latest of them is not a big matter. IDK what is the best freuqency, but increasing to 6 days won't cause any problem. Maybe we can redistribute these saved traffic to the new list. If we set them 8 days, whole added connection by a new list with 4d frequency can be cancelled out. We can further save the number of connection if we merge some of built-in lists. For example, simply merging Privacy and Resource abuse (maybe by !#include), given EP blocks resource abuse too, saves connection of a list with 4d frequency. If we then set its frequency to 8d, we can add a list with 3d frequency without changing the number of total connection. A compromise will still be needed; e.g. if we allow x1.5 # of total connetion:

• Privacy (4d) + Resource abuse (4d) => uBlock filters - Tracking and Resource abuse (8d)
• New list: uBlock filters - Hot fixes (1d, addresses serious circumvention and breakage)

All these can be done by trivial code change with practically no ill-effect to user.

---

_{1: For reference ABP anti-cv list is updated every 1 hour, tho the upstream list itself is not as frequenty updated.}

_{2: Adding a new list with 4d update frequency to the current 5 built-in lists means 6/5 = x1.2 # of connection. If 2 days 7/5 = x1.4 and 1 day 9/5 = x1.8.}

[gorhill]

I like the idea.

[ghost]

But adding a list with update frequency of 1 day means 1.8 times larger number of connection than the current one²
_{2: Adding a new list with 4d update frequency to the current 5 built-in lists means 6/5 = x1.2 # of connection. If 2 days 7/5 = x1.4 and 1 day 9/5 = x1.8.}

Besides the current 5 built-in lists, in terms of abuse of GitHub, there are assets.json and badlists.txt, and uBlock filters request 3 text files (filters-2020.txt ,filters-2021.txt and filters-2022.txt).

Considering this correction, the addition will increase the number of connections by 1.45 times instead of 1.8 times.

A compromise will still be needed; e.g. if we allow x1.5 # of total connetion:

Yuki's suggestions ( uBlock filters - Tracking and Resource abuse (8d) and uBlock filters - Hot fixes (1d)) result in a 1.24x increase in connections instead of 1.5x.

[JohnyP36]

For users this increase (1.2x or 1.5x) must not be a problem I think.

[Yuki2718]

It's not a matter of user, but of those who provide bandwidth. Significant consideration have been put into it.

[krystian3w]

Cloudflare have troubles to provide new bandwidth mainly later older links to optional CDN like jsDelivr.

[uBlock-user]

But adding a list with update frequency of 1 day means 1.8 times larger number of connection than the current one2, and I guess @gorhill is not willing to simply add such a list.

jsDelivr can handle any kind of heavy loads, frequency is not an issue there. Use that server for hot fixes list.

#1645 (comment)