Do respondents become sloppy before attrition?

I am working on a paper that aims to link measurement errors to attrition error in a panel survey. For this, I am using the British Household Panel Survey. In an earlier post I already argued that attrition can occur for many reasons, which I summarized in 5 categories.

1. Noncontact
2. Refusal
3. Inability (due to old age, infirmity) as judged by the interviewer, also called ‘other non-interview’.
4. Ineligibibility (due to death, or move into institution or abroad).
5. people who were always interviewed

In the paper, I study whether attrition due to any of the reasons above can be linked to increased measurement errors in the last waves before attrition. For example, earlier studies have found that item nonresponse to sensitive questions (income) predicts unit nonresponse in the next waves.

For every respondent in the BHPS, I coded different indicators measurement error in every of the last five waves before attrition takes place. My working hypothesis is that measurement errors should increase in the last few waves before attrition takes place, due to decreasing respondent willingness and/or capability to participate.

In the figure below, you find one set of indicators I used. Compliance to the survey does not count as an indicator of measurement error, but I found it interesting to look into nonetheless. I find that respondents are far less keen to do “extra” tasks in the waves before attrition. As measures, of compliance to these extra tasks, I looked at:

1. the respondent cooperation as judged by the interviewer.
2 the proportion of respondents who completes the tracking schedule at the end of the interview, and
3. the proportion of respondents returning a self-completion questionnaire, left after the interview.

In order to be able to interpret the results in a good way, I contrasted the 4 attrition groups with the 5th group of respondents who do not drop out, and are always interviewed.

Compliance with survey task by respondents in last 5 waves before attrition (click to enlarge)

Unsuprisingly, I find that compliance decreases before attrition. Even at 5 waves before attrition, I find differences between the groups, with the “always interviewed” being most compliant, and the later to “refuse” group least compliant. The differences between the groups increase, the closer they get to attrition. Of the groups that attrite, the “noncontacts” and later “ineligibles” do only a little worse than the “always interviewed”. The “refusers” and “inables” have sharply decreasing cooperation ratings, and rates of completing the tracking schedule and returning the self-completion questionnaire. The differences between the groups are not large enough to predict exactly who is going to refuse or become unable to participate, but they can help to identify respondents being at risk.

The next question would be what to do with this knowledge. If a respondent really is unable to participate, there is not so much we as survey practitioners can do about this. Likely refusers may also be hard to target effectively. The rate of noncontacts is to a large degree under the control of survey practitioners, and for that reason, many nonresponse researchers are trying to limit noncontacts. Although refusers may be harder to target than noncontacts, it may be easier to identify potential refusers, and take pre-emptive action, rather than use refusal conversion techniques after a respondent has refused.