3 Pretest Techniques To Reduce Uncertainty For Survey Design

Data collected from survey research is used to make critical business and governmental program decisions. Pretests can reduce the uncertainty of a survey research program by checking participant understanding and capability to complete the survey as intended.  Here are three commonly used pretest formats, and the pros/cons of each:

Types of pretests:

1. In this instance, a survey is launched with a small number of completed surveys.  Checking is typically very technical - is the length of the survey as expected, are people responding to concepts or choice experiments in the expected way, are there high numbers of N/A or Other responses that should be addressed in the design before full launch. 
2. A checkpoint could be a question or verbatim question added specifically for the pre-test that asks the participants to confirm their understanding of a question, or to comment on aspects of their experience taking the survey
3. Pros: Quick, little additional cost, if no changes needed, data can be used as part of final dataset
4. Cons: Limited information, unmoderated
5. Best for: Simple surveys, existing surveys, well understood or well evaluated topics

1. A small number of qualitative participants are tasked with completing the survey, while sharing a video camera. They are instructed to take the survey, speaking aloud their experiences (e.g., “Oh, I’m not sure I can answer this question, I want to say no opinion, but that option isn’t given”). Specific checkpoints can be added on questions of concern, and at the end of the survey for final overall impressions.
2. Pros: Takes less than 24 hours, relatively inexpensive, rich feedback
3. Cons: No opportunity to follow up on feedback, feedback is qualitative in nature, reliant on the participant to provide good feedback.
4. Best for: Detailed feedback when time is constrained, when a small, known number of questions are of particular interest

3.Moderated pretest

1. In a moderated pretest, participants take the survey and then join the moderator for a live discussion. During the discussion, the moderator will display key parts of the survey and discuss with participants. This is especially useful if you have specific questions that you have concern about (e.g., if you are updating a tracking survey, and want to test revised wording). For complex topics, such as surveying medical professionals, the client representative can moderate or observe, providing much needed knowledge.
2. Pros: Able to follow up on feedback, more depth
3. Cons: More expensive, more time consuming
4. Best for: High visibility projects, significant reconstruction of a wave based survey, multi-language surveys.