Survey Research Design

Introduction

Conducting accurate and meaningful surveys is one of the most important facets of market research in the consumer driven 21st century.

Businesses, governments and media spend billions of dollars on finding out what people think and feel.

Accurate research can generate vast amounts of revenue; bad or inaccurate research can cost millions, or even bring down governments.

The survey research design is a very valuable tool for assessing opinions and trends. Even on a small scale, such as local government or small businesses, judging opinion with carefully designed surveys can dramatically change strategies.

Television chat-shows and newspapers are usually full of facts and figures gleaned from surveys but often no information is given as to where this information comes from or what kind of people were asked.

A cursory examination of these figures usually shows that the results of these surveys are often manipulated or carefully sifted to try and reflect distort the results to match the whims of the owners.

Businesses are often guilty of carefully selecting certain results to try and portray themselves as the answer to all needs.

When you decide to enter this minefield and design a survey, how do you avoid falling into the trap of inaccuracy and bias? How do you ensure that your survey research design reflects the views of a genuine cross-section of the population?

The simple answer is that you cannot; even with unlimited budget, time and resources, there is no way of achieving 100% accuracy. Opinions, on all levels, are very fluid and can change on a daily or even hourly basis.

Despite this, surveys are still a powerful tool and can be an extremely powerful research tool. As long as you design your survey well and are prepared to be self-critical, you can still obtain an accurate representation of opinion.

Establishing the Aims of Your Research

This is the single most important step of your survey research design and can make or break your research; every single element of your survey must refer back to this design or it will be fatally flawed.

If your research is too broad, you will have to ask too many questions; too narrow and you will not be researching the topic thoroughly enough.

Researching and Determining Your Sample Group

This is the next crucial step in determining your survey and depends upon many factors.

The first is accuracy; you want to try and interview as broad a base of people as possible. Quantity is not always the answer; if you were researching a detergent, for example, you would want to target your questions at those who actually use such products.

For a political or ethical survey, about which anybody can have a valid opinion, you want to try and represent a well balanced cross section of society.

It is always worth checking beforehand what quantity and breadth of response you need to provide significant results or your hard work may be in vain.

Before you start the planning, it is important that you consult somebody about the statistical side of your survey research design. This way, you know what number and type of responses you need to make it a valid survey and prevent inaccurate results.

Methodology

How do you make sure that your questionnaire reaches the target group? There are many methods of reaching people but all have advantages and disadvantages.

For a college or university study it is unlikely that you will have the facilities to use internet, e-mail or phone surveying so we will concentrate on only the likely methods you will use.

Face to Face

This is probably the most traditional method of the survey research design. It can be very accurate. It allows you to be selective about to whom you ask questions and you can explain anything that they do not understand.

In addition, you can make a judgment about who you think is wasting your time or giving stupid answers.

There are a few things to be careful of with this approach; firstly, people can be reluctant to give up their time without some form of incentive.

Another factor to bear in mind is that is difficult to ask personal questions face to face without embarrassing people. It is also very time consuming and difficult to obtain a representative sample.

Finally, if you are going to be asking questions door-to-door, it is essential to ensure that you have some official identification to prove who you are.

Mail

This does not necessarily mean using the postal service; putting in the legwork and delivering questionnaires around a campus or workplace is another method.

This is a good way of targeting a certain section of people and is excellent if you need to ask personal or potentially embarrassing questions.

The problems with this method are that you cannot be sure of how many responses you will receive until a long time period has passed.

You must also be wary of collecting personal data; most countries have laws about how much information you can keep about people so it is always wise to check with somebody more knowledgeable.

Structuring and Designing the Questionnaire

The design of your questionnaire depends very much upon the type of survey and the target audience.

If you are asking questions face to face it is easy to explain if people are unsure of a question. On the other hand, if your questionnaire is going to include many personal questions then mailing methods are preferable (but may violate local legislation).

You must keep your questionnaire as short as possible; people will either refuse to fill in a long questionnaire or get bored halfway through.

If you do have lots of information then it may be preferable to offer multiple-choice or rating questions to make life easier.

Cover Note

It is also polite, especially with mailed questionnaires, to send a short cover note explaining what you are doing and how the subject should return the surveys to you.

You should introduce yourself; explain why you are doing the research, what will happen with the results and who to contact if the subject has any queries.

Types of Question

Multiple choice questions allow many different answers, including don't know, to be assessed. The main strength of this type of question is that the form is easy to fill in and the answers can be checked easily and quantitatively; this is useful for large sample groups.

Rating, on some scale, is a tried and tested form of question structure. This way is very useful when you are seeking to be a little more open-ended than is possible with multiple choice questions. It is a little harder to analyze your responses. It is important to make sure that the scale allows extreme views.

Questions asking for opinions must be open-ended and allow the subject to give their own response; you should avoid entrapment and appear to be as neutral as possible during the procedure. The major problem is that you have to devise a numerical way of analyzing and statistically evaluating the responses which can lead to a biased view, if care is not taken. These types of question should really be reserved for experienced researchers.

The order in which you ask the questions can be important. Try to start off with the most relevant questions first. Also friendly and non-threatening questions put the interviewee at ease. Questions should be simple and straightforward using everyday language rather than perfect grammar.

Try and group questions about similar topics together; this makes it a lot quicker for people to answer questions more quickly and easily.

Some researchers advocate mixing up and randomizing questions for accuracy but this approach tends to be more appropriate for advanced market research. For this type of survey the researcher is trying to disguise the nature of the research and filter out preconceptions.

It is also a good idea to try out a test survey; ask a small group to give genuine and honest feedback so that you can make adjustments.

Common mistakes when doing the survey research design.

Analyzing Your Results

This is where the fun starts and it will depend upon the type of questions used.

For multiple choice questions it is a matter of counting up the answers to each question and using statistics to ‘crunch the numbers' and test relevance.

Rating type questions require a little more work but they follow broadly the same principle.

For opinion questions you have to devise some way of judging the responses numerically.

The next step is to devise which statistical test you are going to use and start to enter some numbers to judge the significance of your data.

Conclusions

This is where you have to analyze the results. Be self critical whether your results showed what you expected or not. Any survey has flaws in its method so it is always a good idea to show that you are aware of these.

For example, a university represents only a narrow cross section of society; as long as you are aware of this then your results are valid. If your survey gave unexpected results explain the possible reasons for why this happened and suggestions for refining the techniques and structure of your survey next time.

As long as you have justified yourself and pointed out your own shortcomings then your results will be relevant and you should receive a good result.