CIMM identifies best ways to combine STB and Smart TV data

CIMM Study Identifies Best Practices For Integration Of Set Top Box And Smart TV Data

NEW YORK, NY — The Coalition for Innovative Media Measurement (CIMM), today released a study identifying best practices for integration of Set Top Box (STB) and Smart TV data.

The research, which will be presented and discussed at the 10th Annual CIMM Cross-Platform Video Measurement & Data Summit, held virtually on February 3rd and 4th, is designed to assess the strengths and weaknesses of Smart TV and STB data and identify best practices for combining them at the household level. Both datasets are complementary, and the combination comes closer to creating granular nationally representative data sets for linear and streaming TV programming and advertising use cases.

The study was conducted for CIMM by Pre-Meditated Media and Janus Strategy and Insights.

“Set Top Box and Smart TV ACR data sets have quickly gained an influential marketplace position as metric sources for planning, optimizing and post evaluation of TV transactions,” says Jane Clarke, Managing Director and CEO of CIMM. “Simultaneously, demand for analytics that allow advertisers and agencies to precisely plan digital video and CTV on top of linear is rapidly accelerating. As a result, there is a growing industry need to bring standardized second-by-second representative TV datasets into cross-media measurement systems in order to combine them with comparable exposure data for digital video.”

The research was conducted in two phases.

In the first phase, a review of Smart TV ACR and Set Top Box providers was conducted to collect general understanding of the approach of each, including sample size, data captured and reported, data processing rules.

The second phase was to review existing methods used to integrate Smart TV ACR and Set Top Box providers, ranging from matching methods at the device and household level as well as the co-mingled processing of viewing data.

The two phases were then combined to identify best practices. In total, researchers reviewed the practices of 18 companies including leading MVPDs, OEMs/Smart TV Providers and third party integrators.

Phase 1 findings included:

• General application of data beyond TV currency

  The study found that at this time, virtually all applications of Set Top Box and Smart TV ACR are geared for attribution, measurement, optimization and campaign management versus the creation of new audience measurement currency

• Sample size versus representativeness

  Some providers and processors in the analysis make data available from the matched portion of their data sets without modeling the remaining consumers. The rationale offered for this approach is that matched sample sizes are sufficiently large and representative to assess results. But the study found this assumes that demographics account for viewing differences between matched sample and unmatched sample.

  There currently is very limited transparency into the depth of weighting utilized by each provider. Weighting schemes are based on consumer data such as provided by Experian; but generally there is no attempt made to correct for any potential biases in underlying Experian data

• TV data processing, while not standardized across data providers and third-party firms, is far more systematized within individual organizations versus 3-5 years ago, for example

  The researchers noted that procedures for data ingestion, integration and formatting are in place as well as are editing rules. Additionally, algorithmic rules for filling data gaps, e.g., Smart TV ACR distinguishing DVR and VOD, modeling room in house and using in weighting scheme, residence vs. non-residence is still a work in progress

Phase 2 findings focused on identifying five stages of best practices for commingling STB and Smart TV ACR data:

• Stage One – Data set selection – best practices
  • Utilize Set Top Box data sets that span multiple traditional/virtual MVPDs and Smart TV ACR data providers to ensure representativeness of viewer footprint and amplification of complementary measurement properties of both data collection techniques
  • Recognize diversity of household TV access on tuning behaviors that reflect changing landscape of TV viewing and apply consistent definition and sample inclusion of Over-The-Air, Pay TV, and Broadband-Only homes
     
• Stage Two – Establish match and commingling design – best practices
  • Use tuning data from homes with Set Top Box-to-Smart TV ACR device matches to inform calibration of combined data set estimates, including un-matched homes. Three core cells emerge:
    • Set Top Box only, Set Top Box/Smart TV ACR and 3. Smart TV ACR only
       
• Stage Three – Match execution – best practices
  • Deploy high quality matching agent, able to match on postal and IP address
  • Leverage HH device graph to ensure representation of Over-The-Air, Pay TV and Broadband-Only homes
  • Validate match process -Smart TV ACR tuning matched versus unmatched homes; Set Top Box tuning, matched versus unmatched homes; Demographics of matched homes to total U.S.
  • Ask IP match provider questions regarding quality of data records such as recency, churn rate, deterministic vs. probabilistic, life span, etc.
     
• Stage Four – Calibration & Weighting – best practices
  • Key calibrations made to data sets
    • Set Top Box adjustments to Smart TV ACR – # of sets in home, DVR/VOD, backfill reference for ACR signature library
    • Smart TV ACR adjustments to Set Top Box – CTV access/tuning, set-on/set off, on-screen ad exposure
  • Apply weights to four benchmark cells
    • S. demographics, TV access universe, tuning metrics, geographics
       
• Stage Five – Validation – best practices
  • Validate universe and tuning estimates

“There are multiple potential use cases for more granular integrated data sets,” says Gerard Broussard, President of Pre-Meditated Media. “These include for TV audience ratings, campaign planning and optimization, addressable TV campaign activation and measurement, de-duplicated reach and frequency with digital video and CTV and attribution or outcome measurement. The key is to establish the best approach to integration and our analsysis creates a solid framework for that.”

“The feedback we have received in our research bodes well for the future development of granular Set Top Box/Smart TV ACR data sets that support advanced targeting and placement optimization on linear TV,” said Howard Shimmel, President, Janus Strategy and Insights. “What we are seeing is that Integration processes are maturing, enabling more flexibility and potential for standardization; STB and Smart TV ACR data formats already possess some similarities; there is a degree of consistency in metadata already occurring and experimentation is accelerating. All that suggests that the path to integration is taking shape. Our hope is that with the findings of this report that providers will begin to implement the best practices outlined.”