Questionnaire Construction; Including The Use Of Open And Closed Questions

Self-Report Methods Questionnaires

A questionnaire is a set of written questions on a topic on which opinions are sought. Questionnaires are frequently used in survey research in which information is gathered regarding people’s attitudes and beliefs.

Designing questionnaires question choice

Closed Questions (fixed choice):

These have specific, limited answers. Often a statement is given to the respondent and they must choose from several fixed responses. The following example is often referred to as a Likert Scale.

Example:Members of the Royal Family should not receive money that has been raised from the people’s taxes.

Strongly Agree           Agree                Undecided                Disagree               Strongly Disagree

5                               4                             3                                 2                                     1

This kind of question would collect quantitative data.

Evaluation, AO3 of using Closed Questions in Research

Strengths:

(1) POINT: A strength of using closed questions in research is the fact that they will collect quantitative data. EXAMPLE:For example, that encourage participants to circle options (e.g. circle from the list the symptoms of depression you have suffered from over the last 12 months) allows researchers to calculate frequencies and averages. EVALUATION:This is a strength because it means that the researcher can statistically analyse the data, produce graphs allowing for a thorough numerical analysis to be completed on the data.

(2) POINT: A strength of using closed questions in research is that it is easy to replicate the study. This means that since the questions are standardised it is easy to replicate thequestionnaire. EXAMPLE: For example, the same questionnaire can be administered over and over again and the results of this can be compared. EVALUATION: This is positive because it allows for the questionnaires to be assessed in terms of their reliability.

Weaknesses:

(1) POINT:A weakness of using closed questions in research is that they obtain quantitative data. EXAMPLE:For example, participant responses will be summarised in numerical form and therefore may lack detail and depth (e.g. participants are not given the option to elaborate/fully explain their responses. EVALUATION:This is a weakness because the data can be criticised for not accurately representing the complexity of human behaviour.

(2) POINT:Another weakness of using closed questions and questionnaires is that participants may respond in a socially desirable way. This means that participants may give a response they think portrays themselves in the best possible way. EXAMPLE: For example, participants may answer sensitive questions about their weight or sexual activity falsely in order to portray a more desirable image of themselves. EVALUATION:This is problematic because it means the findings are not representative of the truth.

Open Questions

With this type of question the respondent is given a high level of freedom with their answers. Often the researcher simply asks a question and provides space underneath for the respondent to write their answer.

Example: What is your opinion about the Royal family receiving money from the taxpayer?

This type of question would collect qualitative data.

Evaluation, AO3 of using Open Questions in Research

Strengths:

(1)POINT: A strength of using a set of open questions is that these types of questions collect qualitative data. EXAMPLE:For example, open questions allow participants to expand on their answers and provide lots of detail about their behaviour. EVALUATION:This is a strength because open questions collect rich qualitative data which helps researchers develop a better, more in depth knowledge of human behaviour.

(2) POINT: A strength of using open questionnaires in research is that it is easy to replicate the study. This means that since the questions are standardised it is easy to replicate the questionnaire. EXAMPLE:For example, the same questionnaire can be administered over and over again and the results of this can be compared. EVALUATION:This is positive because it allows for the questionnaires to be assessed in terms of their reliability.

Weaknesses:

(1) POINT:A weakness of using open questions in research is that the data collected is in qualitative form. EXAMPLE:For example, open questions allow the participants to fully explain their behaviour in lots of detail. EVALUATION:This is a weakness because when the researcher comes to analysing the data it is very difficult for them to carry out any statistical tests which means that it can be difficult to draw any firm conclusions on the basis of using inferential statistics.

(2) POINT:Another weakness of using open questions and questionnaires is that participants may respond in a socially desirable way. This means that participants may give a response they think portrays themselves in the best possible way. EXAMPLE:For example, participants may answer sensitive questions about their weight or sexual activity falsely in order to portray a more desirable image of themselves. EVALUATION:This is problematic because it means the findings are not representative of the truth.

Designing questionnaires (self-report methods) administering the questionnaire

(1) Postal questionnaires: This involves sending out questionnaires to people through the post. However, this could cause an unrepresentative sample because only people who have time will respond to the questionnaires, this may exclude people who work, have or have full time family commitments.

(2) Magazine and newspaper questionnaires: This involves asking the readers to send in the completed questionnaire. However, this could bring about an unrepresentative sample as only readers of that particular magazine will respond to the questionnaire. This will exclude individuals who don’t read this magazine.

Designing questionnaires ambiguity of questions

Designing the questions for your survey is quite an art. In general, your questions must be clear, simple and mean the same thing to all respondents. The latter means that they must not be ambiguous. Even apparently straightforward questions may be misunderstood.

Poorly designed questions could lower the internal validity of an investigation in order to increase internal validity, it is important that researchers carry out a pilot study in order to ensure that their questions are clear, not ambiguous and can be understood by the participants.

Designing questionnaires (self-report methods) reliability and validity

Reliability  Reliability can be assessed by giving two or more researchers the same questions to analyse. If there is a high level of agreement between researchers, then the questionnaire is seen as reliable. You could also use test-retest to assess the reliability of a questionnaire.

Validity  There is a risk that participants responding to a questionnaire will not do so honestly. Consider the following questions- 1. What is your current weight? 2. Have you ever driven whilst drunk? In these types of questions, due to the fact that they are socially sensitive, participants are more likely to lie/give false information.

If people don’t respond in an honest way then the questionnaire is not measuring what it intends to measure accurately. This means that it will lack internal validity.

Evaluation, AO3 of Self-Report Methods/Questionnaires:

Strengths:

(1) POINT: A strength of closed questionnaires is that they collect quantitative data. This means that questionnaires often collect quantitative data (particularly when closed questions are used). EXAMPLE: For example, a questionnaire using a likert scale would provide the researcher with numerical data to analyse. EVALUATION: This is positive because numbers are easy to analyse and compare.

(2) POINT: A strenght of closed questionnaires is that they can easily be replicated.This means that since the questions are standardised it is easy to replicate the questionnaire. EXAMPLE: For example, the same questionnaire can be administered over and over again and the results of this can be compared. EVALUATION: This is positive because it allows the reliability of the questionnaire to be assessed.

Weaknesses:

(1) POINT: A weakness of questionnaires is that they can lead to social desirability. This means that participants may give answers which they think portrays themselves in the best possible way. EXAMPLE: For example, participants may answer sensitive questions about their weight or sexual activity falsely in order to portray a more desirable image of themselves. EVALUATION: This is problematic because it means the findings are not representative of the truth.

(2) POINT: A weakness of questionnaires is that ambiguous questions can negatively affect participant responses. This means that sometimes questions may be open to different interpretations. EXAMPLE: For example, one respondent might interpret the same question differently to another respondent. EVALUATION: This is a problem because it means that respondents’ answers are not directly comparable and this compromises the internal validity of the findings.