Less Is More in Surveys (Jim Karrh Commentary)

I was just trying to help. Ultimately, I couldn’t bring myself to make it work out. But the experience does shed light on one of the biggest mistakes that organizations make with surveys and other market research tools.
Recently I received an e-mail from a beverage industry publication asking for my participation in an online survey. Normally, such a message in my already overloaded inbox is an invitation to DWG (Delete With Gusto). The enticement for respondents was a one-in-whatever chance at a gift card, and after doing the math I quickly decided the expected value wasn’t a strong enough reason to complete the survey (even if, as they claimed, it would take “only a few moments of my time”).
However, the publication had recently run a news blurb about my employer, Mountain Valley, so with an appreciative heart I decided to go ahead and spend those few moments. After all, the e-mail said that the publisher fully realized that “[my] time is important.”
As it turned out, the purpose of the survey was to test the memorability and potential effectiveness of some of the ads that appeared in the most recent issue of the magazine. A copy of each ad was presented onscreen, followed by a series of questions (did I remember seeing the ad in the magazine, was I overcome with a burning desire to immediately purchase the advertised product, etc.).
Each ad evaluation seemed to take about a minute and a half. I completed the first five, expecting that at some point I would get a warm thank-you and a notice that I had not won the gift card.
Instead, the survey went on.
Ads six through 12 came in succession. I really did want to finish the survey — by now I saw this as more of a personal challenge than as assistance to the publisher — but there were other things to do and, after all, my time is important. By the way, there was no indicator of my progress or how much longer the survey people expected me to toil.
But the survey continued. I was expecting to see reprints of all the classifieds next.
By ad number 15, I’d had it. I bailed, never to return. (There was no “save and come back later” option, either. Too bad.)
There’s a lesson here. When marketing your survey — and you do need to actually market a survey if you want an adequate number of the right respondents — think of the customer. In a world where Americans say time is their most precious and limited resource (at least according to surveys), chances are you will need to err on the side of less.
Require less time. Inflict less hassle. Let respondents in on what you are asking and how long it will likely take. Otherwise, the people who do respond to (and ultimately complete) the survey will by definition tend to be those who have, well, nothing better to do. And are those people really representative of your market?
Many executives make a related mistake in their survey efforts: They treat the exercise as a fishing expedition. In my prior experiences as a marketing consultant, I advised dozens of executives on conducting research (mostly surveys). Most of them, unfortunately, began the process without a clear plan for using the results in decision-making. On more than one occasion, clients would have already drafted some proposed questions; when I asked the purpose of a question, I would hear, “Well, we just thought that might be interesting to know.”
My advice: Unless you will use the results of a survey for specific tracking purposes (such as evaluating changes in consumer attitudes over time) or as input into a pending decision (such as whether to develop a new product or modify advertising messages), then there’s little reason to ask the questions. You will waste time and money.
You won’t make any friends among your survey respondents, either.

(Jim Karrh, Ph.D., is chief marketing officer of Mountain Valley Spring Co. of Hot Springs. E-mail him at [email protected].)