Some General Principles for Survey Item Construction
( First Page > Composing Items > This page)
Each survey question provides a unique set of challenges, but there are a few general principles for question construction.
  1. Sometimes it is best to be blunt
    Some topics are best handled by a simple, straightforward question. Being roundabout or metaphorical only confounds the data. This works best when the issue is fairly public anyway, and there would be no conflict to the individual in expressing their opinion.
    1. The organizational chart connects me with the right people to do my job well.
    2. Other departments involve my work group appropriately in their planning and decision-making.
    3. My manager gives us the information we need about issues and events in the company.
  2. Be subtle
    Some issues are best approached from the side. Especially if the topic might reflect poorly on the respondent, or require them to acknowledge negative feelings toward peers or their boss.
    1. How do you think most of your co-workers would evaluate your manager?
      NOTE: This is much softer than asking "How would you evaluate your manager?" In an environment that is somewhat intimidating or overly hierarchical, the more subtle version may work better.
    2. Which of the following images would best describe the typical staff meeting in your department?
      [ ] Mud wrestling?
      [ ] Family dinner?
      [ ] Railroad roundhouse?
      [ ] Sovereign holding court?
      [ ] Murder in the Dark?
      [ ] Football huddle before the snap?
      [ ] Pre-game pep talk?
      [ ] Other _____________________________
      NOTE: This style of question has always provoked lively and productive debate during feedback sessions. But be sure that you have some good options in the mix. In one client "Sovereign holding court" was the clear winner in the data. That prompted a valuable review of how staff meetings are conducted.
    3. Overall I am treated like which of the following here at {Company}?
      [ ] A trusted colleague
      [ ] A cog in the machine
      [ ] A replaceable part
      [ ] An asset
      [ ] A family member
      [ ] Other _____________________________
      NOTE: Using the "Other" category is quite useful here. For one client, the majority of respondents wrote something in the optional category, and the most common metaphor was scatological. It was a dramatic wake-up call to executives who had no idea employees felt so poorly treated.
  3. Make it honorable to be bad
    Some questions tap into the respondent's less than honorable preferences or viewpoints. The framing of the question has to make it acceptable to be shortsighted, disloyal, selfish, stubborn, or less than fully competent. Here's some examples:
    1. Sometimes market conditions can force all of us to make difficult choices. How likely are you to become a non-union shop if the economy does not improve in the coming year?
    2. Long range planning is always problematic in a changing market. How confident are you of achieving your goals for this year?
    3. Everyone has to consider their own personal career as well as the success of the company. How likely are you to be working with this company one year from now?
    4. Moving ahead in your career sometimes means taking on a job that is somewhat outside of your comfort zone. How well do you think you have mastered the skills required for your job as of now?
  4. Always know how you will understand a negative answer.
    A good test for any question is whether you are absolutely confident that you know what a "No" would mean. Consider the following examples:
    1. Does gender have a positive, neutral, or negative impact on career advancement in this company?
      NOTE: The scale on this question ran from "Positive" through "Neutral" to "Negative". But what does "negative impact" mean? Does it mean the same thing from a male as it does from a female?
    2. The material I learned in this course will help me achieve my group's business goals.
      NOTE: Questions like this presume too much. If I answer "No", does it mean my group has no business goals? Or that I didn't learn anything? Or that what I learned won't be useful?
    3. My manager provides critical feedback and support.
      NOTE: If the respondent disagrees, does it mean their manager fails to provide critical feedback? Or support? Or both? The flaw in this question is that critical feedback and support are hard to provide together. If your question has 2 themes, make sure they are closely related; otherwise break it into 2 questions.
  5. Use open-ended questions to enrich data you've already collected, rather than as a primary data source.
    Open ended questions (like "What changes would you suggest for our new strategic direction?") are a common survey strategy, and they're commonly misused. The best use of open-ended questions is to qualify or expand on data you've already collected in some other question. If you're using an open-ended question as a primary data source, you should probably spend more effort in survey design (for example, conducting interviews or focus groups).
    1. Open-ended questions are extremely costly to process and analyze. Imagine getting a paragraph or 2 from hundreds of respondents!
    2. Open-ended questions are a good way to uncover the emotional tone behind the numeric data. Apathy, cynicism, and anger could all produce the same numerical score, but the comments section might reveal the underlying emotional tone.
    3. A good use of an open-ended question is to close off the survey with a question such as "Is there another topic we didn't cover in the survey questions that you'd like to raise?" Responses to that question will give you a solid clue about whether your survey included the right topics or not.
Back to Top