Validity is the test or instrument researcher is using actually measures what she/he needs to have measured (Salkind, 2012, p. 123) while reliability of a measure concerns its ability to produce similar results when repeated measurements are made under identical conditions (Bordens & Abbott, 2014, p. 126). In order to be reliable, a measurement must be dependable, replicable, and consistent. Particularly, the measurement should reduce random error. Random error occurs in two ways: (1) when an individual randomly guesses a response, perhaps because the question is poorly worded or has misleading directions, and (2) when an individual accidentally chooses a response unintentionally (Ruel, Wagner III, & Gillespie, 2016). Hence, in order to avoid such problems, researcher will do the pilot test to see how dependable and consistent it is, and explain the purpose of the instrument (questionnaire) in detail to the respondents.
The relationship between reliability and validity is straightforward and easy to understand (Salkind, 2012, p. 127). For instance, a test can be reliable but not valid, but a test cannot be valid without first being reliable. In other words, reliability is a necessary, but not sufficient condition for validity (Ibid.). Therefore, the content of each instrument (LCSESQ and MSESQ) is validated by specialists and experts who have experiences in respective areas of expertise ranging over 10 years. Moreover, the pilot-test will be conducted by distributing 17 questionnaires of each (LCSESQ and MSESQ) to TESOL undergraduate students who are not parts of the sample to determine reliability of the instrument and to ensure an accurate survey measurement. According to Isaac and Michael (1995, p. 101) and Belle (2002, p. 11), 17 respondents are sufficient to determine the reliability of the questionnaires.
A reliability test for the questionnaires is checked after the pilot study data was collected. The questionnaire has a Cronbach’s alpha values analysed by IBM SPSS as follows: learning challenges towards spoken English skills, σ = 0.90 and motivation towards spoken English skills, σ = 0.80 which are considered ideal (Pallant, 2007, p. 95).
According to Dörnyei and Csuzér (2012, p. 89), they observed that pilot-test does not only help to check reliability, but it also serves to help researcher as follows:
- Adapt and adjust the questionnaire in order to eliminate ambiguous, too difficult/easy, or irrelevant items.
- Improve the quality and clarity of each item wording and instruction.
- Finalise the questionnaire’s layout – make it easy to go through.
- Rehearse the administration procedures.
- Test the expected findings whether they potentially emerge from the data. o Reduce or increase the completion time of the questionnaire if necessary. o Double-check that mistakes are eradicated completely from the instrument.
In summary, a questionnaire that is checked by the experts and tested in pilot-test would definitely be reliable and validate to collect the data needed for the research.
