measurement and nonresponse error in panel surveys

I am spending time at the Institute for Social and Economic Research in Colchester, UK where I will work on a research project that investigates whether there is a tradeoff between nonresponse and measurement errors in panel surveys.

Survey methodologists have long believed that multiple survey errors have a common cause. For example, when a respondent is less motivated, this may result in nonresponse (in a panel study attrition), or in reduced cognitive effort during the interview, which in turn leads to measurement errors. Lower cognitive abilities and language problems might be other examples of common caused that lead to either nonresponse or measurement error. Understanding these common error sources is important to know whether our efforts to reduce 1 survey error source are not offset by an increase in another one. It follows from the idea that good survey design minimize Total Survey Error

Studying the trade-off has proven to be very difficult. This is because nonrespondents are by definition not observed. So, we never know how nonrespondents would answer questions, and how much measurement error is included in those answers. We can only observe measurement errors for respondents, but can not compare these to the potential measurement error of nonrespondents.

Hypothetical continuum of timing of survey response

To overcome this problem, most methodologists have compared ‘early’ respondents (people who respond very quickly in the fieldwork period) to ‘late’ respondents (those who only participate after being reminded for example). The idea behind this, is that the probability of response is:

a) a linear continuum from very early response on the one extreme, and nonresponse on the other.
b) that hypothetically, nonrespondents could be converted into respondents if extreme amounts of efforts are used to do so (Voogt 2005 showed in a small-scale study in the Dutch locality of Zaandam that this is actually possible)

So, the idea in summary is that late respondents can serve as a proxy for information about nonrespondents. However, that assumption is not likely to be true in general, if ever.

In my project, I will try to overcome this problem, that we never have measurement error estimates for nonrespondents. I use longitudinal data and Structural Equation Modeling techniques to estimate measurement errors for nonrespondents in the British Household Panel Study, compare them to respondents, and link them to potential common causes of both type of errors. See this presentation for more details on this project