For my team’s smartphone project, we decided to put together a survey, which gave us feedback on what our users had, liked, and disliked about their smartphone.
Upon looking across the data we received I started to notice that the responses we get are only related to the questions we ask. This may sound self explanatory, but it’s really quite surprising. Not only how you word the question, but how you choose the method of response determines the data you finally get. For example, some of our questions could be ambiguous, such as: most of our question were tailored to smartphone users, even though we asked at the beginning of the survey, “Do you own a smartphone?”
Gary Barber had an interesting blog post on this topic: http://manwithnoblog.com/2010/02/17/user-surveys-do-it-right-or-not-at-all/
Although our survey wasn’t long or multiple pages, I can easily see a few ways that we could have made our survey more effective.