Assessing the representativeness of new data sources

The big data revolution is beginning to dramatically open up the types of data sources that can be used to measure and study behaviors, such as pornography use. The subscription data used by Edelman (2009) represents the type of large datasets that will increasingly become available to scholars in their research. An important first step in using this type of proprietary data will be assessing the degree to which the data from a single provider is representative of the general population of interest. In this section, we provide a framework assessing the representativeness of a dataset by comparing it to the patterns observed from another data that is known to be nationally representative or by comparing it to a combination of other data sources that collectively are likely to represent the true underlying pattern of behavior.

In Table 1 we list the top ten and bottom ten states for pornography use based on each of the four sources: subscription data, Pornhub, NFSS, and Google Trends. Mississippi is one state that ranks in the top four states in pornography use across all four datasets and Idaho consistently ranks near the lowest rates of any states across most of the measures. In contrast, other states such as Arkansas and Utah rank in the top ten along some measures but in the bottom ten along other measures. These results suggest that identifying which state seems to have the highest rates of pornography use based on a single data source can be a bit problematic.

Table 1. Rank Order of States Based on Four Different Data Sources Controlled for Broadband Internet Access.

[NFSS 2012] Rank . . . 50 . . . Utah

[Google Trends 2013-2014] Rank . . . 46 . . . Utah

[Paid Subscription 2006-2008] Rank . . . 1 (highest) . . . Utah

[Pornhub 2013] Rank . . . 51 (lowest) . . . Utah