For anyone carrying out their own market research online, there are a number of pitfalls to watch out for. Chris Brookes and Gill Wales of the Independent Consultants Group offer some in-depth guidance
When you're considering doing a survey yourself, don't forget that all surveys aren't the same. It is fine to conduct certain types of online survey yourself, for example:
- customer feedback on a recent purchase experience;
- incidence checks (to find out who is buying a particular product or doing a particular thing);
- PR stories;
- employee surveys on non-sensitive subjects such as their views of the staff restaurant.
How to write a good questionnaire
The purpose of a questionnaire is to collect information that you can compile into statistics. This means that the questions have to be clearly understood without any risk of people interpreting the same question in different ways.
If the information you collect is to have any value, it's essential that questions are not leading and that people feel able to give clear and truthful answers.
A poorly constructed questionnaire will generate vague responses that either don't answer your objectives or, worse still, give you unreliable data.
Some complicated types of research need interviewers to ask the questions and/or complex analysis systems to process the data. But even the most straightforward questionnaire design benefits from the knowledge of some basic rules.
The importance of sampling
The type and number of people you include in your survey (your sample) is even more important than writing a good questionnaire. You can write the best questionnaire ever created, but if you end up with responses from an unrepresentative sample of people, or you invite responses from people to whom the survey topic isn't relevant, then you may have collected data that is misleading or just plain wrong.
For example, you might be asking about a recent purchase experience, but if the people you invite to take part in the survey include lapsed customers, people who haven't yet bought from you, or people who haven't received their order yet, that part of your sample will find it impossible to respond.
You also need to take care how you use and store the data, bearing in mind the Data Protection Act and, from May 25 2018, the General Data Protection Regulation (GDPR).
Before you write a questionnaire, think about what type of person you need answers from. Then work out how you are going to reach them. Check mailing lists for relevance, currency and completeness, and check what recent contact they're had with your company, including other departments. You'll also need to ensure that you're not contacting people too often, inappropriately or inconsistently.
If you're posting the survey on a website, does that website have the type of traffic you need? Also, it is usually important to know the survey response rate - that is, what percentage of the people who saw the survey actually participated.
The validity of the research results depends not only on the number of people who take part, but also how well those people represent the audience you are researching. Does your survey sample broadly share the same characteristics (age, occupation, gender, products purchased) as the population it is intended to represent?
Making sense of the findings
It's important to accurately analyse and communicate your findings. You will probably have both open-ended (qualitative) and closed (quantitative) questions to analyse. Think about the person or people who will be acting on the information; what will they need to know in order to take action?
Above all, you will have to organise and interpret the data so that conclusions are reliable and the common pitfalls of working with survey data are avoided.
The journalists' Who, What, When, How and Why questions are a useful guide when you are preparing a report:
- Why was the research done?
- What does your audience really want to know?
- Who did you survey, and who needs to know the results?
- When do they need the results?
- How best to report the results? A PowerPoint presentation, Word document, or verbal briefing?
- So what? Do the survey results tell you enough to make firm recommendations for action, or do they raise new questions, needing more research?
Are all surveys suitable for DIY research?
No. There are three reasons why you might be better off hiring a professional to help with your survey - technical complexity, sensitivity and business risk.
Technical complexity
If you need to carry out technically complex projects such as market segmentation, pricing policy, or attitudes and motivation, then you will need specialist assistance.
Sensitivity
Different survey topics have different levels of sensitivity. Sometimes it is because the subject matter is highly personal, such as personal hygiene, personal finance, or religion, or what someone thinks about their job or their boss.
Sometimes it is because of perceived objectivity: will the respondent feel they can trust you with the information? Often, an independent third party is important in order to reassure respondents of confidentiality.
In addition, some projects are politically sensitive internally to the organisation conducting them.
Risk
Research can generally be categorised as strategic (because it relates to the organisation's long-term strategy) or tactical (because it relates to smaller, single issues).
Strategic research projects could have a major impact on your organisation's future, so there is greater risk attached to them. Generating unreliable or biased results could have serious consequences. For strategic research surveys it's important that professional researchers are always involved.
Six top tips for DIY research
- Start with your end-goal in mind - as you are writing the questionnaire, consider how you will present the findings.
- Keep your survey as short and simple as the topic allows.
- Doing a survey isn't just about writing questions: your sample and how you communicate your findings are equally important.
- Involve (at least) one other pair of eyes at each stage, to help review, pilot, or check understanding.
- Liaise with people who can help you, either internally or externally.
- Don't be afraid to ask for help.
