The Makings of a Monstrous Survey

Since it’s almost Halloween, I thought I’d post something scary. Sort of. In my line of work, I see some scary stuff..meaning scarily misdesigned. Okay, perhaps as scary as the Stay-Puff Marshmallow Man from Ghostbuster (was anyone really ever afraid of him???!), meaning not very scary at all. These surveys fall into the “wow, what were they thinking?” kind of scary.

1. The Thing That Wiggled And Squirmed Survey.
In years past, I had the misfortune to assist a non-profit group who created a monstrous 356 question survey (my screams went unheard) that they wanted to make web-based. This was the beast with 356 legs. It wiggled, squirmed AND all of its questions were required! This survey tortured its victims for an hour and 35 minutes long before they could submit it (or it ate them..not sure). Last I heard, this group disbanded their survey because of low response rate. I suspect that many people tried to take this survey and eventually saw that struggle was futile and just gave up. Then the survey ate them. Keep this beast under control by limiting your number of questions to 20 and under!

2. The Chinese Question Torture Survey. It’s the constant drip that eventually drives you insane. Some surveys can be the same way. Take something small and repeat it enough that it drives you to madness. I took a survey from a well-known marketing software company just the other day that had me begging to make it stop. This was perhaps one of the worst usages of skip logic that I’ve ever see. I would answer a question such as “what is your favorite color?” I would choose “blue” and then the next screen (drip), would say “so your favorite color is ‘blue'” (drip), so how would you rate your preference for “blue” (drip), submit and the next screen (or do I mean “scream”?) says “so you said that your favorite color is blue (drip)..how does blue rank on a scale of one to ten compared to red, green, and orange?” (drip drip drip). To avoid this horror, keep your questions short, pin-pointed and remember that if you waste your survey participants’ time, they may NOT come back for your next as a vengeance!

3. The Exorcist Survey. This kind of survey is surely possessed. When a survey starts out with the following question:

Please rate our excellent service:
Excellent, Great, Good, Not Good

you can sense an unseen hand guiding to the survey participant to act, to answer in a way that makes you wonder–hum, he doesn’t seem like himself today! Okay, probably not the work of Satan but more likely just skewed, biased responses but in my world, that’s darn SCARY!! Keep your questions neutral and your data will stay away from the dark side..

Speaking of scary movies, I saw “The Devil’s Backbone” by Guillermo del Toro. I can’t handle scary movies but this one was perfect to give me a chill while not making me think something was hiding under the bed last night. So fire up your DVD player after the kids go to bed (yours or the ones who egged your house), put on a good scary movie, have some warm cider, and keep the dog nearby..happy haunting!!

It’s Uptime!

This week the Marketing Sherpa published a few pieces on surveying. Some good pieces but one criteria that they forgot to mention is uptime. I was reminded of uptime recently when I received an email from one of our competitors explaining to their customers that they’d be taking their servers down for maintenance. Any surveys on their system will be unavailable during this time.

To me, this is a cardinal sin in software development. Any one who knows me knows that I’m all about the user and planned outages for maintenance are unacceptable in my book. The ultimate user inconvenience. We cannot know when a survey participant will respond to our surveys so they need to be available ALL THE TIME! Otherwise, we are undoubtedly losing an opportunity to collect data as we cannot assume that someone will take time to fill out the survey later. In this day and age, there is no excuse for “planned” outages.

At Beeliner, redundancy is a good word..as in “our servers are redundant” not, “Amanda, you already said that. You’re being redundant.” We never make our service unavailable due to maintenance since we have a team of servers. We take one down at a time and the others cover for the server that is being upgrading. This makes me appreciate the time and care that was put into the construction of our systems. Thanks Heiko and the tech team!

The Best Time and Day to Send Email

A recent article from the Marketing Sherpa showcased a company that ran diagnostic tests to determine which day of the week and which time of day yielded more response. I’m always fascinated by these studies, keeping in mind that results with vary company by company. The idea that the very time you send your survey as well as the very day you select is something we’ve discussed in the past. Some key findings from this study:

• Tuesdays beat Mondays and Wednesday for better response rates. No mention what happens if Monday is a holiday but I would guess that in a holiday week, everything shifts by a day. No mention of Thursday or Friday in this study. For most business uses, I’d say never send your survey on the weekend..it will get lost in the clutter that most people find in their inbox first thing on Monday mornings.
• Mornings are max. Sending at 9am performed 15.63% better than 4p.m. and 9.4% better than 12p.m. for maximum responses. 12p.m. performed 6.9% better than 4p.m.

Our guess as to why 9am performs better than the rest of the day? In the morning, most people make a point of spending time to check their email and take care of things before diving into their agenda for the day.

Radio Buttons and Checkboxes: did you put a square box where you need a round button?

If I had to name the most common survey design error that I see, it is confusion around radio buttons and checkboxes and how they pertain to choice questions. If you aren’t sure what radio buttons and checkboxes are, don’t worry..you are undoubtedly familiar with them even if not by name. Radio buttons are simply those round buttons that accompany a single choice selection, allowing you to select just one answer. Checkboxes accompany a multiple choice selection and allow you to select multiple options. If you’ve ever filled out a form online or even used a computer, you’ve used radio buttons and checkboxes. They are such a part of the way that we interact with our computers, that many of us don’t even think about these interface elements anymore than we would taking a breath of air! However, just like breathing, we may forget about them when we begin to design our surveys. Using checkboxes for single choice questions, especially rating scales, is a common mistake. It can be confusing to your survey participants and it can mess up your data collection. After all, if you’ve asked your survey participant to rate on a scale of “1 to 5,” you don’t want them to pick 3 AND 4, right? Whenever you’re not sure which type of question to use when designing your survey, check out the sample survey on our homepage which showcases many types of questions and how they can be used! See sample survey.

The A to Zs of Employee Surveys Webinar Audio Available

Beeliner Surveys

On Tuesday May 26th, 2009 we presented a webinar “The A to Zs of Employee Surveys” with CreativeSuccess.com CEO Gail McMeekin! Listen to it here (approximately 1 hour)