How “test your adblocker” websites can harm users and the adblocker ecosystem

This post was written by Shivan Kaul Sahib (Lead for Privacy Engineering).

Brave provides best-in-class privacy protections, including robust third-party ad and tracker blocking. As demand for privacy grows, more websites have emerged to compare web browsers and extensions. While some of these comparison tools, such as https://privacytests.org and https://coveryourtracks.eff.org, are valuable resources for privacy-conscious users, others can be significantly flawed. This post explores how some adblocking test websites—those that claim to assess adblocker effectiveness—fall short due to poor testing methodology, and even harm the ad and tracker blocking ecosystem through misleading results.

The problem with adblocker testing websites

Adblocker testing sites can be misleading—or even harmful—for a number of reasons. We’ve outlined the four major issues below.

Low-quality tests

First, the quality of these test websites is generally low. While well-intentioned, many adblocking test websites check for blocked domains that are non-existent or not even used for advertising or tracking. Additionally, there is often no—or poor—justification provided for why a particular test exists.

For example, consider a test website that checks whether your adblocker blocks HTML elements with specific class names, such as .box.banner_ads.adsbox. Most testing sites will provide no rationale for why an adblocker should block these elements. It is possible that adblockers have identified more effective selectors to block actual ad-serving elements, but without a clear explanation for the test’s criteria, it’s impossible to evaluate its validity. Essentially, the test arbitrarily checks for blocked elements without any justification for why they should be targeted by any reasonable adblocker.

Savvy users could easily add a custom filter (such as ##.box.banner_ads.adsbox)to adblockers like Brave to artificially pass these tests. However, doing so would provide no tangible benefit, and create unnecessary rules that could inadvertently break websites. Testing, if done carelessly and erroneously, does more harm than good to privacy tools, and ultimately, users.

Tests often do not emulate real-world blocking

Testing websites often fail to emulate real-world ad delivery and tracking practices. For example, some will issue network requests to base domains without specifying full URL paths, and expect them to be blocked, as pointed out by a filter list maintainer. This approach does not reflect how trackers or ad servers operate in practice. Adblock maintainers carefully craft their rules to create the minimum required blocking to avoid breaking website functionality—a nuance these tests ignore entirely.

Tests ignore advanced protective capabilities

Advanced adblocking capabilities, such as resource replacement, are often overlooked in these tests. Resource replacement refers to how adblockers specify replacing a known tracking script with an innocuous version of the script. This is useful for preserving website behavior while still blocking the privacy-harmful script. Because these features aren’t easily measurable by simple network request tests, they’re excluded from a testing website’s scoring mechanism, which can lead to underreporting of an adblocker’s true effectiveness.

Tests cannot account for domain-specific rules

To reduce the risk of website breakage, adblockers often use domain-specific rules (rules that only apply on a particular website) rather than generic rules that apply on all websites. For example, a request to usertrack.js on the website search.com might be something that an adblocker wants to block, because it knows that that usertrack.js on that website is used for cross-site tracking. However, on a different website like shoes.com, a script named usertrack.js could be a legitimate script used for keeping track of user preferences like night mode or theme. Test websites cannot test for this nuance because their evaluations are confined to their own domain. Thus a simple overlap in script naming could lead to scoring mistakes.

Why testing websites can be misleading for users

Fundamentally, these tests are unable to differentiate between adblockers effectively. Most adblockers rely on the same community-maintained filter lists for their blocking rules, with minor deviations. Brave, for instance, directly supports and maintains the well-regarded EasyList projects, and publishes research and techniques related to ad and tracker blocking. This shared foundation allows filter list maintainers to work together and adapt quickly, especially against well-resourced advertising and tracking companies.

This means that testing websites that naively measure “is your adblocker blocking requests to tracker.com?” have limited value, since the underlying rules are mostly the same across major adblockers. Worse, these tests encourage adblockers to manipulate results to avoid misleading users. Both uBlock Origin and AdGuard, two major adblockers, have added workarounds to pass tests (uBO workaround, AdGuard workaround) so as to avoid confusing and alarming their users.

Brave has actively reached out to—and attempted to help—the authors of many test websites to improve their tests in the past (here are just a few examples), but we’ve had to deprioritize such efforts due to fundamental issues in the testing methodology (as described in this post) and the fact that low-quality tests continue to be added.

How adblockers should be measured against one another

Adblockers should (and often do) differentiate themselves in areas like extension-based vs native support, performance, advanced adblocking features, and ethical practices. It’s these differentiators that adblock comparison sites—and users—should focus on.

Extension-based vs. native support

With Google’s Manifest V2 deprecation, adblocking Web extensions face existential threats to their capabilities. At the very least, Manifest V3 severely hamstrings adblocking extensions by imposing a cap on blocking rules and banning background pages while requiring service workers. Brave’s Rust-based adblocking engine, integrated directly into the browser and not dependent on any extensions, remains unaffected by anything Google might require for extensions. This ensures that Brave Shields will continue to offer a superior ad and tracker blocking experience.  Brave will even support some Manifest V2 extensions that users might want to use in the interim before fully switching to Brave Shields as their main adblocker.

Performance

Efficient matching algorithms ensure minimal impact on browsing speed and system resources while blocking ads and trackers. Brave’s Rust-based adblocking engine is built into the browser, thus allowing us to make deep changes in both the browser and the adblocking code which ensures best blocking results and maximum flexibility. We’re actively working on improving our engine’s performance especially as it relates to memory usage.

Advanced adblocking capabilities

Brave constantly ships new adblock features like procedural cosmetic filtering, which allows for fine-grained control over ad and tracker blocking, and to expand the kind of ads we can block for our users.

Ethics

Unlike extensions that participate in “pay-to-play” programs, Brave does not accept payments to unblock certain advertisers or trackers. This ensures that user privacy and experience remain top priority.

In general, adblock tests do more harm than good

We applaud the intention behind these adblocker testing websites. The online third-party based advertising and tracking ecosystem has metastasized to a point that even US intelligence agencies use adblockers internally to counter the threat of malvertising. Brave has been outspoken in its condemnation of third-party advertising and has proposed alternatives like Brave Ads that prioritize the user, not ad-tech third parties. But while well-intentioned, adblocking testing websites currently do more harm than good.  By promoting arbitrary and low-quality metrics, these tests mislead users, force adblockers to adopt unnecessary workarounds, and detract from meaningful advancements in the field.

In addition, while adblockers are a crucial tool, privacy-focused browsers like Brave can protect users in ways that standalone adblockers cannot. Read our privacy updates blog for more details on how Brave ships privacy features to protect and enhance your Web browsing experience. Brave will continue to work with legitimate testing sites like https://privacytests.org and https://coveryourtracks.eff.org and with the larger open source filter list and adblock communities. It’s only through a consistent testing methodology and agreed-on sets of common privacy harms that we’ll be able to improve Web privacy for all users.