Leah Melani Christian, Nicholas L. Parsons, and Don A. Dillman Designing Scalar Questions for Web Surveys Sociological Methods & Research 2009 37: 393-425. SMR 37:3 was a Special Issue on Web Surveys.
This paper explores how the visual design of scalar questions influences responses in web surveys. We present the results of five experiments embedded in two web surveys of university students. We find that consistently presenting the positive end of the scale first did not impact responses but increases response times. Displaying the categories in multiple columns influence how respondents process the scale and increase response times. Separating the midpoint, “don’t know” option, or endpoints spatially does not impact responses when the visual and conceptualmidpoint align. Removing the graphical layout of the scale influences responses when lower numbers indicate more positive categories and increases response time. Finally, response times are longer for polar point scales with numeric labels, but there are no differences in responses. Overall, our results suggest that the visual design of response scales impacts measurement, butthat some manipulations produce larger and more significant differences than others.
Key Words: web surveys • visual design • scalar questions • measurement