Test Categories & Interpreting Test Results
Behavioral analysis is a core component of our analysis, and the tests we run fall into several categories:
- Security: these tests highlight risks to the user’s personal safety, such as malware/mal-advertising, browser hijacking via pop-unders, and cross-site scripting attacks. We’re always growing the number of tests!
- Discoverability: these tests all come down to one question: “could a person reasonably discover this content organically?”
- Transparency: Adherence to industry transparency standards, like: presence of an ads.txt file, TCF strings properly encoded, no personalization prior to banner acceptance, etc…
- Adherence to legal requirements, such as: presence of a privacy policy, presence of a CMP banner when visiting from EU, etc…
- Bid requests contain accurate information, and are not obfuscated / hidden in any way from observers
- User Experience: these tests relate to things like Core Web Vitals, ad-density measurement, and page weight & busyness.
- Advertising: Relates specifically to ad-industry standards & platform policy violations which govern the frequency, positioning, and quantity of advertisements.
- Brand Suitability: These tests are generally to do with content that advertisers may be averse to. We’re looking for adult-oriented sites, pirated content, Made-For-Advertising signals, and more.
Interpreting Test Results
Every result has either a “risk” or “quality” designation, but not all tests are equally important. Results designated “High Risk” are so bad that even a single one has huge implications on the overall site quality. This stands to reason, because “High Risk” test results include things that are banned / looked-down on across almost every major DSP and SSP, like: piracy, browser hijacking, malware, and MFA site classification.
The current High Risk tests are:
- Peak Density Average >= 50%: This is flagged when a site has >=50% of the screen occupied by ads at least once during most crawls. This is only true of about 1% of all sites we track, making this an extreme outlier in terms of peak density.
- >=35% Average Snapshot Density: Over 35% of the screen is occupied by ads at any given time. Average snapshot density above 35% is highly anomalous; only about 1% of sites exhibit this behavior
- Adult Content: This site contains explicit sexual imagery & themes, or is observed loading code specific to adult ad-networks
- Piracy: This domain hosts (or is connected to a site that hosts) movies, tv, or print media without permission from the copyright holder. This includes laundering fronts, which are not obviously pirate sites at first visit.
- MFA: This domain is confirmed to source its traffic inorganically, and it has an excessive ad experience. These sites have been manually verified to contain the most excessive clickbait.
- Template Site: Template sites are characterized by generic content, and non-unique design. They often appear like MFA sites, but they don’t always have paid traffic sourcing. They are typically associated with SEO spam.
- Ad Clutter: This site has been manually confirmed to present an excessive count of ads, and/or an excessive ad-density. Sometimes this manual review is necessary due to attempts to baffle automated tests. These sites can often be confused with MFA sites, but they don’t purchases the majority of their traffic.
- Purchases Black Market Traffic: This site obtains a large amount of traffic from pop-networks that hijack browsers without user intent
- Hijacks the Browser with Pops or Redirects: Using this site exposes visitors to significant risk by means of forced pop-unders and redirects
- Embedded: This site is loaded surreptitiously under the content of other pages, causing errors in referral detection, and creating invalid advertising events
- Tag Spoofer: When visiting this page, tags are loaded that declare the publisher URL as somewhere other than where the user actually is.
- Spoofed Tags: When visiting other pages, tags are loaded that declare this site as the publisher URL, even though the user never visited this site. Tag URLs may differ from the visited page for legitimate reasons, but very few
- Regularly Loads Malware: Our crawlers detected malware on the majority of days we visited this site. While it is possible that this represents incidental exposure to mal-advertising, the frequency with which we encounter suspicious scripts should give pause. Either the publisher is knowingly integrating suspicious scripts, or they have an immediate need for creative verification tech integrated on-page.
- Majority Slow CrUX Scores: This site has serious user experience issues, with poor results in 50%+ of the Chrome user experience metrics we capture. Please refer to the User Experience tab for more detailed information.
Aside from “High Risk” results, the rest of the interpretation is largely up to your own working model of site quality. Our hope is to allow you to enforce your organizations own quality guidelines by providing you objective metrics that other companies struggle to affordably provide at scale. If you need a consultation on establishing best practices, we’re happy to do a consultation with your team.