The traditional web survey is dead

Why we should throw out most of what we know on how to visually design web surveys

In 2000, web surveys looked like postal surveys stuck onto a screen. Survey researchers needed time to get used to the idea that web surveys should perhaps look differently from mail surveys. When I got into survey methodology in 2006, everyone was for example still figuring out whether to use drop down menus (no), how many questions to put on one screen (a few at most), let alone whether to use slider bars (it’s not going to reduce breakoffs).
We now know that good web surveys don’t look like mail surveys. Proponents of ‘gamification’ in surveys argue that survey researchers are still underusing the potential of web surveys. Perhaps they are right. So, while web surveys are ubiquitous, and have been around for almost 20 years, we still don’t exactly know to to visually design them.

The next challenge is already upon us. How are we going to deal with the fact that so many surveys are now being completed on different devices than computers? Touch screens are getting more popular, and screens for smartphones can be as small as 4 inches. People are nowadays using all kinds of devices to complete surveys . The rules for how a questionnaire should look like on a 21 inch desktop monitor are in my view not the same however as for a 5 inch Iphone.

Just take the following picture as an example. It was taken from the Dutch LISS panel in 2013, before they came up with a new and much better design for mobile surveys. Yes this is still what many people still see when they answer a survey question on a smartphone. Is this the way to do it? Probably not. Why should there be a ‘next’ (verder) button at the bottom of the page? Can’t we redesign the answer scale so that we use more space on the screen?
 

Small touchscreens ask for a different design. Take the picture below. Tiles are used instead of small buttons, so that the entire screen is used. There is no next button; once you press a tile the next questions shows up. To me this looks better, but is it really? Only proper (experimental) research can tell.

I also still see lots of problems with adapting our visual design to smartphones. Where to put the survey question in the example below? On a separate page? What if respondents tick a wrong answer? And, what if some respondents get this question on a PC? Should the questions look the same to minimize measurement differences, or be optimized for each device separately? Web surveys are dead. Mixed-device surveys are here to stay.

I think there are lots of opportunities for exciting research on how to design surveys for the new era of mixed-device surveys. For anyone interested, I recently edited an open-access special issue on mixed-device surveys together with Vera Toepoel, where different authors for example study:

- why using slider bars is a bad idea for (mobile) surveys
- the effects of device switching within longitudinal surveys ( study 1 and study 2 ) on data quality
- different possible visual designs for implementing mixed-device surveys , and their effects on data quality

These studies are just the beginning. Much more is necessary, and it will take trial, error and probably another 15 years before we have figured out how to design surveys for mixed-device surveys properly.