The following is a guest post from our friends at Unwind Media, with statistical insights provided by Edward Krueger, Chief Data Scientist at DeepSee:
Solitaired.com faces a challenge familiar to many publishers. While our platform and engagement stats are strong, we are still a young company and don’t have the brand recognition from advertisers to drive meaningful direct sales. As a result, our near-term goal is to make our ad inventory as compelling as possible for SSPs and subsequently DSPs and their advertisers.
The concept we were interested in testing is straight forward:
In addition to requiring that our ad units are viewable when refreshing, what happens if we also ensured that ads would not be shown to sessions that have gone idle?
This means for ads to refresh the placement would need to be deemed in-view and the user will have to have performed an action on-page within the last 60 seconds such as a click or scroll event.
In-view refresh is industry standard so adding an engagement requirement takes that a step further. It’s a simple change we hypothesized would increase viewability, decrease total bid requests and have minimal negative impact to topline revenue.
The concept for this test was driven by a focus on measurement and efficiency. While the casual gaming category has awesome websites, some have fallen under appropriate scrutiny for arbitraged traffic and poor ad experiences and we wanted to create as much distance from those stereotypes as possible. This, in addition to our +99% direct traffic footprint and strong user engagement stats, would help us stand out.
We partnered with DeepSee to evaluate test design and validate results. DeepSee is an industry leader in assessing publisher ad quality and while they typically focus on identifying bad actors, this was an opportunity to shift that focus to a feature that could potentially establish best practices of high-quality ad inventory.
About Us
Solitaired.com is a casual gaming platform with over 500 games. The most popular are classics like solitaire, freecell and a word game called Phrazle. Stats we’re proud of include +400% YoY session growth, 7 games played per user session, an 8x monthly return rate and over 3 million games played daily.
We’re a small business powered by user growth and monetized by open exchange ad revenue. To balance both, we try to present an ad footprint we believe is reasonable and we experiment with to improve. On desktop our standard layout is two display units with a dismissible outstream (placement=5) video player, on tablet we have two display units, and mweb has a max of two units depending on device type.
Thoughts heading into the test
I was confident based on Solitaired’s direct traffic profile and engagement metrics that we have very active users, but we had not run a similar test prior so it wasn’t clear what metrics would be affected. However, we had no reason to suspect we had idle users and our domain level viewability was already strong at around 84%.
We defined idle as no activity for 60 seconds. We settled on 60 seconds vs. 30 seconds because we knew some of our users think about their next move or guess within a 30 second timeframe. Extending it a bit beyond our 30 second in-view refresh trigger felt warranted.
Test setup and considerations
This test was set up as a before-after comparison. Although we typically favor A/B testing, we elected not to run this as an A/B test because we thought it would significantly increase complexity while muddling the positive impact of improved viewability to the buy-side, which is often measured on the macro ad unit vs. impression level.
While we recognized that seasonality could impact revenue, we were confident that core metrics like viewability and bid requests wouldn’t be affected and the intention wasn’t short term CPM driven.
This test went live on July 5th for display. We were initially hesitant to include our dismissible outstream unit since the user has the option of closing that unit at any point during gameplay (making it harder to measure). However, we elected to include it since it could be argued this feature is even more critical to video than display.
Technical Implementation
For purposes of this test, we defined a user interaction as one of the following events: a click, a scroll, or a touchstart.
In addition to monitoring these events, an ad must meet viewability requirements (as defined by GAM via their impressionViewable event combined with our own viewability monitoring in-browser), regardless of when it is rendered or refreshed.
With each user interaction, we reset our 60-second user interaction timer. On any display refresh opportunity, we check to ensure that the user interaction happened within the last 60 seconds. If not, no calls are made to SSPs, nor are calls made to GAM to trigger a new auction and ad render.
Our outstream video runs on its own auction cycle timer and this is subject to the same 60-second interaction time frame as display.
Once a user returns to interact with the page (using any of the above events, or with a window focus event) display ads are re-enabled, and the outstream auction cycle is restarted.
Metrics tracked
- Total impressions – The total number of ad impressions, paid and unpaid, served to a user per game.
- Paid impressions – The total number of ad impressions monetized above price floors per game.
- Unpaid impressions – The total number of ad impressions that either didn’t receive a bid or fell below established price floors.
- Viewability – The change in viewability by device type before and after implementation.
- Ad requests – The change in total ad requests by domain before and after feature implementation.
Results
The following metrics describe the period between July 6th and August 5th 2022:
Conclusion
We consider this a successful test and have elected to keep it live across 100% of our inventory. As we saw no decrease to paid impressions, this is do-no-harm to immediate performance that puts us in a better position to advocate for spend and inclusion in always-on PMPs and high viewability packages.
Similar to the standardization the industry has seen with refresh on-view, we believe there’s benefit for both buy and sell side with the adoption of refresh on-idle. The more buyers continue to value and reward publishers for increasing viewability and request efficiency, the greater benefit this will have for publishers and with better performance will come more rapid adoption.
Have any thoughts after reading? Come and join the conversation with us on linkedin or twitter!
