Do great research: Strategies to ensure quality survey results
Sponsored by The W Group
High-quality research is an important tool that supports effective decision making for services, programs, and infrastructure in communities. While many municipal staff have experience in designing surveys and do a good job at it, there are often some minor (and yes, sometimes major) errors in the wording, design, or collection plan. If not corrected, these errors can end in misleading and even disastrous outcomes.
Managing hundreds of surveys per year exclusively for Canadian municipalities, The W Group has developed some effective strategies to help their clients better understand customers’ and residents’ needs and desires, providing the tools for evidence-based decision making.
Specific to municipalities, here are some observations to keep in mind when you want to “do great research.” The observations fall into one of two categories: barriers to survey participation and barriers to great survey data and clear results.
Barriers to Survey Participation
These barriers are quite specific to municipal or community surveys and all impact how many residents complete the survey. This is the quantity side of the survey challenge. Getting a sufficient quantity of residents from all parts of your community to complete the survey is key to you being able to use the survey data for decision making. Here are the top barriers to residents participating completely in your survey:
- doing minimal project promotion due to assumption of high-level public interest;
- not considering other projects within the municipality that may be vying for residents’ attention;
- using communication tools and methods that are non-specific to your target audience;
- designing a survey without consideration for mobile users (more than 65 percent of surveys are completed on a mobile device); and
- not providing a clear definition and disclosure of why personally identifiable information is being collected from respondents.
Barriers to Great Survey Data and Clear Results
The worst moment in research is realizing your survey has produced bad or unusable data. You can avoid this by ensuring your survey questions are well constructed. This can be more challenging than it appears. Common barriers to getting great data and results from your research can include:
- muddying a clear message by using subject-specific language and acronyms;
- not providing mutually exclusive (single answer) or exhaustive options;
- asking for personal information (e.g., household income or age) at the beginning of the survey, resulting in a higher number of incompletes;
- ignoring the visual learner in (most of) us by using descriptors when a good visual would say a thousand more words;
- using an unengaging survey title and project description, thereby decreasing engagement and survey completion rates; and
- starting a survey with a complex question that deters respondents from completion because they assume the entire survey will be involved and lengthy.
When doing community-based research, it’s important to remember that, to get high-quality data, the building blocks of great surveys are the questions we ask and how we frame them. MW
Kent Waugh has been providing professional research services to municipalities for the past 35 years. As Managing Partner of The W Group and ClikHere.ca, he works exclusively with municipalities to deliver great research surveys.