AAPOR 2014

Big data and new technologies to do survey research. These were in my view the two themes of the 2014 AAPOR conference . The conference organisation tried to push the theme ‘Measurement and the role of pubic opinion in a democracy’, but I don’t think the theme was really reflected in the talks at the conference. Or perhaps I have missed those talks, the conference was huge as always (> 1000 participants).

The profession of survey research is surely changing. Mick Couper last year argued that the ‘sky wasn’t falling’ on survey research, but it is evolving. Big data may potentially replace parts of survey research, especially if we don’t adapt to new technologies (mobile), and learn to use some of the data that are now found everywhere. Big data and survey research in fact have the same basic goal. To extract meaningful information out of datasets (big data) or people (survey research), and use that to inform policy making.

Big data can certainly be useful for policy-making. Out of the 10 or so presentations that I have seen at AAPOR, most were however just talking about potential possibilities over using big data to inform policy makers.
What was in my opinion missing at AAPOR were good case studies that showed how big data can replace survey research and provide valid inferences. I have seen many good earlier examples when it comes to predictions at the level of an individual using big data. When Amazon tries to recommend me books that relate to a book I have previously bought, I find these useful and accurate predictions of what I really like. In politics, voter registration records data can help politicians target likely voters for their party, as the 2012 Obama campaign showed .

But when it comes to aggregating big data to the level of the population, big data is often in trouble (the Obama election campaign is an outlier here, as they collect data on the whole population). Survey research has relied on the principle of random sampling from the population to draw inferences, but for big data, coverage and nonresponse errors are often unknown and unestimatible for the convenience samples that big data ususally are. Paul Biemer made this point in an excellent talk .

Most of the other big data presentations at AAPOR to me were either in the category ‘bar talk’ - anecdotes without a scientific empirical strategy - or just talked about the potential of big data. And don’t get me wrong: I do think that big data are very useful, especially if they cover a late proportion of the population (e.g. voter records), or if the goals is prediction at the level of an individual.

The other conference theme seemed to be mobile surveys. With Vera Toepoel, I gave a presentation on this topic, which may be the topic of a next blogpost. Here, I think survey researchers are much better equipped to deal with the challenge mobile devices pose. I saw many excellent presentations on questionnaire design for mobile surveys, and selection bias.

Finally, this is just my conference take-away. Some other bloggers ( here here ) seem to have a slightly different view on the conference. Probably this is due to the fact I have only seen 1 out of the 8 presentations given at any time. So be sure to check their posts out if you want to know more about the conference.