panel conditioning

In late august of 2011 I attended the Internet Survey Methodology Workshop. There were people from academia, official statistics and market research agencies there. One of the issues discussed there has had me thinking since: the topic of panel conditioning. Some people seem really worried that respondents in panel surveys start behaving or thinking differently because of repeated participation in a survey.

Panel conditioning is closely linked with the issue of  ‘professional’ respondents. These are respondents who know exactly how survey researchers design surveys, and use this knowledge to get most out of the survey (in terms of reward-schemes) against the least time possible.

Many market research firms throw out respondents after some time, mostly a couple of years, and then refresh their samples. But is this necessary? If so, after what time do respondents become conditioned? And for what topics is conditioning most problematic?

Several studies from the 1970s focused on voting behavior in election panel studies. They found that respondents who were asked before a general election aboyut their voting behavior were 10-15% more likely to vote than respondents who were only asked about their voting behavior after the election. I wrote about exit-polls earlier; panel conditioning might be one of the reasons why Internet-panels do so badly at predicting election outcomes. Many other studies have focused on panel conditioning: for attitudes, cognitive abilities, knowledge, marital satisfaction and consumer behavior. Use google scholar on ‘practice effect’, ‘reactivity’, ‘panel conditioning’, ‘test-retest effect’ and you’ll see what I mean.

Overall, the findings suggest that panel conditioning may indeed be problematic, but not in all studies, or for all people. I have some ideas on the circumstances that lead or do not lead to conditioning effects (topic saliency, interval between measurements, frequency of measurement), but none of the studies systematically analyses potential causes for conditioning effects. I am hoping to add some work on this issue in the next years. If anyone know of interesting panel studies that are confronted with panel conditioning effects, let me know…