- Olga Wolska-Płonka
How to prepare an effective survey
Why do we need surveys?
“Understand your customers’ pain points and then establish processes to reduce customer effort. […] Conduct a survey of your customers to identify the key aspects that you can improve to provide a better customer experience."
Shane Barker, Digital Strategist
Surveys are a powerful tool for gathering information and gaining insights into the thoughts, opinions, and preferences of individuals. They provide a means to understand and measure public sentiment, consumer behaviour, market trends, and more. Surveys enable organizations to make informed decisions, tailor their products and services to meet customer needs, identify areas for improvement, and drive positive change. By collecting data directly from respondents, surveys offer a representative snapshot of a target population, facilitating evidence-based decision-making and fostering a deeper understanding of various social, economic, and cultural aspects. They provide a voice to individuals and allow their perspectives to be heard, contributing to inclusive and participatory processes. Ultimately, surveys empower organizations, and researchers to shape strategies that align with the needs and desires of the people they serve.
However, writing a survey is an art of its own. To make it simpler for you, I will present you some tips on how to prepare them effectively.
Things to remember about respondents
As we can read in the research paper “ International Handbook of Survey Methodology”, the person that takes our survey has several tasks to fulfil during this time. They may impact the way respondents answer the questions, so it is very important to be aware of them. Those tasks are:
It is crucial for respondents to interpret the questions in the same way to ensure consistency in the responses. Therefore, it is essential to minimize any potential for misunderstandings. Ambiguity has no place in the questions. They must be clear and straightforward.
When answering questions, people need to retrieve information from their memory. However, this can be difficult if they don't have the necessary information or if they can't recall it. To ensure accurate survey results, it's important to provide all relevant information and select respondents who are likely to have the needed information in their memory. For example, asking about something that happened 10 years ago may not be appropriate.
Our aim is to assist respondents in effectively conveying information through articulate responses. This requires an accurate answer format, clear and concise, and exclusive and exhaustive options for closed-ended questions. For rating questions, a direct question format is recommended.
People answering our survey will not always want to provide an answer to our questions. As reported by Locander et al (1976), when official records of drunk driving were compared to arrest reports, there was considerable underreporting. A similar situation could be seen in the research of Cannel, Oksenberg and Converse (1977), conditions regarded as embarrassing, were reported significantly less. There are three main reasons why respondents would change their answers:
Firstly, they may fear revealing information related to illegal activities like drug use or stolen money that could lead to prosecution.
Secondly, they may desire to present themselves in a more favourable light by altering their answers to appear more intelligent or socially popular.
Lastly, respondents may feel pressure to distort their answers to achieve a more accurate classification if the literal response could potentially lead to an incorrect conclusion.
Tips on preparing the survey
Define a clear, attainable goal for your survey When creating a survey, it is crucial to have a clear and attainable goal in mind. This goal will serve as the foundation for the entire survey design and help you focus on gathering the necessary information. Start by asking yourself what specific insights or data you hope to gain from the survey. Are you trying to understand customer satisfaction, collect demographic information, or assess employee engagement? Defining a clear goal will help you craft relevant questions and ensure that the survey is focused and effective.
Assure of confidentiality When preparing a survey, assuring confidentiality is crucial to encourage respondents to provide honest and accurate responses. Clearly communicate the purpose of confidentiality, distinguishing between anonymity and confidentiality, and use reassuring language to convey your commitment to privacy. Explain data handling procedures, and compliance with data protection regulations, and consider including a confidentiality statement and consent. By assuring confidentiality, you create a safe and secure environment for respondents to share their thoughts, ensuring the reliability of the data and building trust for future participation.
Don’t let your survey get too long Long surveys tend to result in low response rates and a higher likelihood of respondents abandoning the survey halfway through. To overcome this, it is essential to keep your survey concise and avoid unnecessary questions. Determine the key information you need to gather and focus on those aspects. Consider the time it takes for respondents to complete the survey, as well as their attention span and willingness to provide thoughtful answers. Keeping the survey short will increase completion rates.
Keep it brief, simple, and specific When creating survey questions, it is important to keep them brief, simple, and specific. Clear and concise questions are easier for respondents to understand and answer accurately. Avoid using technical jargon or complex language that might confuse or frustrate participants. Use simple and direct language that is easily comprehensible by a diverse range of respondents.
Pay attention to the order of questions The order of questions in a survey can significantly impact respondents' perception and their ability to provide accurate and thoughtful answers. Generally, it is advisable to start with easy and non-sensitive questions to help respondents get comfortable and engaged. Place more complex, challenging, or personal questions towards the end when participants have already established a level of trust and commitment to completing the survey. This approach reduces the risk of early survey abandonment and ensures that respondents are more likely to provide genuine and considered responses to sensitive topics.
Allow respondents to respond “not applicable (N/A)” In some cases, respondents may encounter questions that are not relevant to their situation or experience. To accommodate such scenarios, it is essential to include a "not applicable" or "N/A" response option. This allows respondents to indicate when a question does not apply to them, avoiding the need for them to guess or provide inaccurate information. Including this option maintains the integrity of the data by ensuring that responses are valid and meaningful.
Don’t as leading questions. Leading questions can inadvertently influence respondents' answers by suggesting a particular response or biasing their perception of the topic. It is crucial to avoid leading questions that may steer respondents towards a desired outcome. Instead, aim for neutral and unbiased wording that allows respondents to express their genuine opinions or experiences. Review each question carefully and consider how it might unintentionally guide respondents' answers to ensure the survey remains objective and unbiased. Example: “How much did you enjoy our services?” The question already assumes the respondent enjoyed the service.
Don’t use absolutes Using absolute language, such as "always," "never," or "everyone," in survey questions can lead to inaccurate responses and misrepresentation of data. Absolutes tend to oversimplify complex situations and may not account for exceptions or variations in individuals' experiences. It is best to avoid absolutes and instead use more moderate or nuanced language that accurately reflects the range of possibilities or perspectives. Example: Do you always eat breakfast? (Yes/No) The question will force most of the respondents to answer “No”, as everyone in their life has a situation when it happens that they didn’t eat breakfast.
Don’t ask double-barrelled questions Double-barreled questions combine multiple inquiries into a single question, making it difficult for respondents to provide clear and distinct answers. This can lead to confusion and inaccurate data. Each question should address only one specific aspect or topic to ensure clarity and allow respondents to provide focused responses. If you have multiple points to cover, break them into separate, individual questions for better accuracy and understanding. Example: How satisfied or dissatisfied are you with the pay and work benefits of your current job? - Two questions, about pay and about work consolidated into one. Should be separated.
Include a short introduction and a time estimated Providing a brief introduction at the beginning of the survey can help set expectations and establish the purpose of the survey. It is an opportunity to explain the importance of respondents' participation and reassure them about the confidentiality or anonymity of their responses. Additionally, including an estimated time for survey completion helps manage respondents' expectations and allows them to allocate the appropriate amount of time for the task. Example: Hi there! Our company aims to develop the best possible products for our customers, and we need your opinion to make this happen! We are very curious to hear your thoughts about our new feature, your input is very valuable for us. The survey should only take 5 minutes, and your responses are completely anonymous. If you have any questions about the survey, please email us: bigcompanyl@email.com Thank you for your time!
Be thoughtful about demographic questions Demographic questions, such as age, gender, or ethnicity, can provide valuable context for analyzing survey responses. However, it is essential to be thoughtful and considerate when including these questions. Ensure that the demographic information you are collecting is relevant to your survey's goals and purpose. Avoid asking overly personal or sensitive demographic questions, and provide options that are inclusive and respectful of diverse identities and backgrounds.
Make survey questions mandatory only if a response is necessary Consider making survey questions mandatory only when the response is crucial for your analysis or decision-making process. Mandatory questions can increase the respondent's burden and may lead to incomplete or inaccurate responses if they feel compelled to provide an answer they are unsure about. Carefully evaluate each question and determine if the data you will obtain from it is essential. If a question is not critical, consider making it optional to reduce respondent fatigue and increase completion rates.
Write questions respondents can understand One of the fundamental aspects of survey design is ensuring that the questions are written in a way that respondents can easily comprehend. Use clear and concise language, avoiding technical terms or jargon that may confuse participants. Make sure the questions are free from ambiguous or vague phrasing that could lead to different interpretations. Consider the target audience's educational background, cultural diversity, and language proficiency to ensure the questions are accessible to all respondents. Pilot testing the survey with a small group can help identify any potential confusion and refine the question wording for better clarity.
Preview and test your survey before sending Before distributing the survey to your intended audience, it is crucial to thoroughly preview and test the survey. Previewing allows you to review the survey layout, question order, and formatting to ensure it appears as intended. Pay attention to question branching or skip logic if you have incorporated it to verify that the flow is logical and functioning correctly. Testing the survey also involves submitting test responses to check for any technical issues or errors in the survey platform. By conducting these checks, you can identify and address any problems proactively, ensuring a smooth and error-free survey experience for your respondents.
Test or pilot your survey and survey platform beforehand Before distributing the survey to the intended audience, it is crucial to conduct thorough testing or piloting. This ensures that the survey functions correctly, all questions are clear and understandable, and there are no technical issues with the survey platform. Piloting the survey with a small group of representative respondents can provide valuable feedback and help identify any areas that need improvement or adjustment before the survey is launched on a larger scale.
Conclusion
Creating an effective survey requires thoughtful planning and attention to detail. Surveys are valuable tools for gathering insights and data, but their success relies on clear and concise questions, proper consideration of respondent tasks and potential biases, and an understanding of survey design principles.
By following these guidelines and considering the needs and experiences of respondents, you can create surveys that yield accurate and meaningful results, ultimately enabling informed decision-making and driving positive change.
Here is a list of example questions that can inspire you, or that you can use in your survey. (It’s a page in Notion you can copy to your account.)