Research Methods (1)

Questionnaire Surveys

Questionnaire surveys are one very important method of gathering original data for analysis in social and market (business) research.  However, the process of preparing and designing a questionnaire, and of analyzing the data is not as simple as it may seem.

1. Identify the aim of the research – what information are you trying to gather from the survey?
Consider carefully the purpose of the survey, so that you can decide which questions are most important.  Do not simply begin by writing a collection of questions.  Without a definite purpose, the survey will not be successful.
2. Identify the population and sample
The population is all the members of the group you are interested in [ e.g. all “English major students in China” may be the population for a survey on the attitudes of English major students to XXXX  .  A sample is a sub-set of the population that is chosen because to access all members of a population is prohibitive in time, money and other resources.  The sample should be representative of the population – it should normally be chosen at random, or selection processes explained carefully to ensure no bias.  The bigger the sample, the more representative it should be, but it should be big enough to eliminate individual differences as far as possible.
3. Decide the method you will use for collecting replies

There are two basic methods – using an interviewer (a person to ask the questions) [either in person, or by telephone etc.] or a written (or e-mail) questionnaire in which the respondent writes directly (called a self-administered questionnaire).  It is important to explain in either case the purpose of the questionnaire, and what it will be used for.

4. Questionnaire design

a.       determine the questions
i. Relevance Questions should be relevant to the data which is required.  Irrelevant questions (for example asking about boyfriend/girlfriend in a survey about job preference) should not be included.

ii. simplicity Questions should be easy to understand and answer, otherwise people will not bother to complete the questionnaire.  Interesting questions will help motivate people to answer.  Often, a question may not be a question at all, but a statement with which the respondent can either agree or disagree.

b.      select the type of question

i.        open vs. closed questions An open question is one to which respondents can reply in any way they want.  Closed questions give a range of options from which the respondents should choose.  Open questions can get more individual, accurate and detailed answers, BUT the data is often difficult to analyze later, so wherever possible, closed questions are preferable.  Closed questions should give a complete list of likely responses, and if uncertain, a choice for “other” – please specify.

[e.g. What do you think is the most important thing for university students today?]

ii. single vs. multiple answers A single answer is normally preferable, as data can be more easily analyzed, but in some cases, multiple answers may be accepted.

[e.g. How do you travel to university a car b bus c bike d train]  Can be turned into a single answer by adding most often

iii. ranking answers Where answers are ranked according to attitudes, importance etc. (1 as most important).  These can often be difficult for respondents to choose, so should be kept to a minimum

iv. rating answers  These are very popular in social research – giving a rating (1-5, 1-10 etc.) to your feeling about a certain statement.

c.       Question wording

i. clear and unambiguous avoid jargon and words respondents may not be familiar with.  Remove words like “recently” which are vague and depend on the interpretation of the respondent.  Replace with specific words (in the last two weeks).

ii. avoid double questions e.g. Do you think that students should eat less and exercise more?

iii. avoid double negatives They cause confusion for respondents: Are you against a ban on smoking?

iv. precise answers are easier to analyze and more useful: Give your age on Sep.1 2006

 v. avoid “leading” questions. Questions that tempt the respondent to give a certain answer will invalidate the data.  “Do you agree with President Hu Jintao that…”

vi. contingency questions Sometimes, a question may only be applicable to some of the respondents.  You can solve this by making use of layout, so that respondents do not waste time reading sections not related to them. E.g. (Do you have a boyfriend / girlfriend?  If yes, How often do you see each other? Etc. etc.)

d.      design the sequence and layout

i. order of questions Note that the order in which questions are asked can affect the answers given, so think about this carefully!  It is usually best to begin with the most interesting questions, and leave questions about demographic data to the end.

ii. layout of the page Do not clutter the page.  Make sure there is only one question on each line, and that the information is indented so that a respondent will not miss a question by mistake.

iii. space for answering boxes [ ] or ( ) are best for single responses.  A scale can be used for rating or ranking.  Try to be consistent wherever practical.  Give careful instructions on whether to circle, cross out or tick, cross etc. so that data analysis is made easier.

5. Check and test the questionnaire to find any problems
It is always best to test the questionnaire first to identify any problems BEFORE you give the questionnaire out to respondents.  It is much easier to correct mistakes before the survey, and in many cases, mistakes can completely invalidate the research.  Pilot tests are vital to eliminating as many variables as possible.

6. Carry out the survey
Make sure delivery is reliable, and collection is as easy for respondents as possible.  On collection, number each questionnaire so that data analysis can be carried out easily and accurately

7. Analyze the data

We will deal with this in another class.

Your job now is (in pairs) to design a questionnaire survey that we are going to give to 2^nd year students.  I want you to take this activity seriously, and to prepare a questionnaire ready for use before next week.  I will collect them, select the most appropriate questions and then give the survey to Aaron to give the second year students.  We will then collect the survey and do some basic analysis to elicit some results.

The purpose of the research is to find out something about “How Chinese students view themselves and their position in the modern world”