Checklist research instrument example
IntroductionWe will start with a few key operational definitions. Surveying is the process by which the researcher collects data through a questionnaire (OLeary, 2014). A questionnaire is the instrument for collecting the primary data (Cohen, 2013). Primary data by extension is data that would not otherwise exist if it were not for the research process and is collected through both questionnaires or interviews, which we discuss here today (OLeary, 2014). An interview is typically a face-to-face conversation between a researcher and a participant involving a transfer of information to the interviewer (Cresswell, 2012). We will investigate each data collection instrument independently, starting with the interview. InterviewsInterviews are primarily done in qualitative research and occur when researchers ask one or more participants general, open-ended questions and record their answers. Often audiotapes are utilized to allow for more consistent transcription (Creswell, 2012). The researcher often transcribes and types the data into a computer file, in order to analyze it after interviewing. Interviews are particularly useful for uncovering the story behind a participants experiences and pursuing in-depth information around a topic. Interviews may be useful to follow-up with individual respondents after questionnaires, e.g., to further investigate their responses. (McNamara, 1999). In qualitative research specifically, interviews are used to pursue the meanings of central themes in the world of their subjects. The main task in interviewing is to understand the meaning of what the interviewees say (McNamara, 2009). Usually open-ended questions are asked during interviews in hopes of obtaining impartial answers, while closed ended questions may force participants to answer in a particular way (Creswell, 2012; McNamara, 1999). An open-ended question gives participants more options for responding. For example an open-ended question may be, How do you balance participation in athletics with your schoolwork (Creswell, 2012). A closed-ended question provides a preset response. For example, Do you exercise? where the answers are limited to yes or no (Cresswell, 2012). Must-knows before the interviewInterviewer must be:
Different Types of Interviews
Structured Versus Unstructured
Sequence of Questions
Both Creswell and McNamara highlighted very similar points about conducting interviews. McNamaras literature is less descriptive, but more simple and concise. Another author who has come up consistently in the interviewing literature is Kvalve, whose literature is much more intensive and broad. These three authors are all very prominent in the interview research literature. Conducting the InterviewThese are the steps that are consistent in the literature on conducting interviews in research (Creswell, 2012; McNamara, 1999):
Strengths
Weaknesses
QuestionnairesQuestionnaires have many uses, most notably to discover what the masses are thinking. These include: market research, political polling, customer service feedback, evaluations, opinion polls, and social science research (OLeary, 2014). Formulating a QuestionnaireStarting OutBell & Waters (2014) and OLeary (2014), each offer clear checklists for creating a questionnaire from beginning to end. By comparing the two, we have created a comprehensive list. Bell starts by reminding the researcher to obtain approval prior to administering their questionnaire, then to reflect on what our question is and whether this is the best method to obtain the intended information (Bell & Waters, 2014). OLeary (2014) suggests that you operationalize concepts in the beginning and define the measurable variables. Prior to writing your own questions, OLeary (2014) would have you explore existing possibilities in order to adapt previous instruments rather than reinventing the wheel. At this point, both authors have you write your questions. Forming questionsBell & Waters (2014), utilizes Youngman (1982)s Question Types:
Bell & Waters (2014), highlight a plethora of potential difficulties in wording your questions, including ambiguity and imprecision, assumptions, memory, knowledge, double questions, leading questions, presuming questions, hypothetical questions, offensive questions, and questions covering sensitive issues. It is imperative that you check for jargon within your language and return to your hypothesis or objectives often to decide which questions are most pertinent (Bell & Waters, 2014). Bell & Waters (2014) and OLeary (2014) seem to disagree on the next step; while OLeary would focus next on the response category, Bell would have you look further into the wording of the questions. Following OLeary (2014)s logic, we decide now whether to use open or closed questions, considering how the category will translate to different data types. Closed response answers include: yes/no, agree/disagree, fill in the blanks, choosing from a list, ordering options, and interval response scales. Any of the three standard scaling methods, (Likert, Guttman, and Thurstone) may be used where appropriate (OLeary, 2014). Bell & Waters (2014) suggest you check your wording at this point. OLeary (2014) goes into detail to point out problems with questions such as ambiguity, leading, confronting, offensiveness, unwarranted assumptions, double-barrelled questions, or pretentiousness. Questions to avoid according to OLeary are those that are:
Ordering Questions / Appearance and layoutBoth authors emphasize thoughtfulness about the order of questions, considering logic and ease for respondents. OLeary (2014) goes into further detail regarding issues with organization and length; too lengthy and respondents are less likely to complete the questionnaire. He also suggests researchers avoid asking threatening, awkward, insulting, or difficult questions, especially in the beginning of the questionnaire. Bell & Waters (2014) takes a more broad view of the aesthetics of the questionnaire; leaving spaces for legibility, limiting the overall numbers of pages, and considering the impression the document leaves, to highlight a few examples. Write InstructionsClear and unambiguous instructions for respondents are emphasized by both authors (OLeary, 2014; Bell & Waters, 2014). This step is followed by a layout, or rearranging of questions, in both descriptions, likely because this is the best time to review once the questions and other writing is complete. OLeary (2014) warns researchers to use professional and aesthetically-pleasing formatting, as well as to be organized in order to attract respondents and to lower the probability of making your own mistakes (in repeating questions, for example). OLeary (2014) offers final instructions to include a cover letter that describes who you are, the aim of the project, assurances of confidentiality, etc.. Bell & Waters (2014), however, offers further steps. Sample & Pilot TestingBell & Waters (2014) go into further detail regarding response rates and ensuring you have a representative or generalizable sample, which we believe is irrelevant to this article. More pertinent steps would be to pilot-test your questionnaire with preliminary respondents (even family and friends) and follow-through to preliminary data analysis in order to ensure your methods are effective, making adjustments accordingly (Bell & Waters, 2014). OLeary (2014) lists six steps in a typical pilot test:
DistributionBell & Waters (2014) briefly consider distribution methods; they emphasize the need to ensure confidentiality, to include a return date, to formulate a plan for bounce backs via email, and to record data as soon as it arrives. OLeary (2014) lists typical methods: face-to-face, snail mail, e-mail, and online. Bell & Waters (2014) highlight the advantage to administering your questionnaire personally, as it enables the researcher to explain the purpose of the study and increases the probability of receiving completed questionnaires in return. The authors go on to emphasize the value of online methods. In particular, they mention Survey Monkey as the most popular and versatile survey tool available (Bell & Waters, 2014). OLeary (2014) suggests sending out reminder letters or E-mails in order to increase response rate and the speed of response. AnalysisBell & Waters (2014) and OLeary (2014) disagree once again with respect to the analysis. OLeary (2014) suggests collecting the data as soon as possible, whereas Bell (2014) suggests the researcher merely glance through the responses prior to coding and recoding, if time allows. Both methods have merit, as the researcher must consider the time they have available, as well as the amount of data they are working with in order to make a logical decision. WeaknessesOLeary (2014) offers some concerns in using questionnaires as a research tool, as they are time consuming, expensive, and sampling is difficult. OLeary (2014) asserts that questionnaires are notoriously difficult to get right and they often do not go as planned. StrengthsOLeary (2014) suggests some obvious strengths for this research method, as administering a questionnaire allows the researcher to generate data specific to their own research and offers insights that might otherwise be unavailable. In listing the additional benefits of questionnaires, OLeary (2014) suggests that they can:
Considerations for the MethodCohen et al. (2013, p.394) offer special considerations for administering questionnaires within an educational setting:
ReferencesBell, J., Waters, S., & Ebooks Corporation. (2014). Doing your research project: A guide for first-time researchers (Sixth ed.). Maidenhead, Berkshire: Open University Press. Cohen, L., Manion, L., Morrison, K., & Ebooks Corporation. (2011; 2013; 1993). Research methods in education (7th ed.). Abingdon, Oxon; New York: Routledge. doi:10.4324/9780203720967. Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Los Angeles: Sage. Kvale, S., & SAGE Research Methods Online. (2008). Doing interviews. Thousand Oaks; London: SAGE Publications, Limited. McNamara, C. (1999). General Guidelines for Conducting Interviews, Authenticity Consulting,LLC, Retrieved from: http://www.managementhelp.org/evaluatn/intrview.htm OLeary, Z. (2014). The essential guide to doing your research project (2nd ed.). London: SAGE. |