SavedAdd your journal

Journals / Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi / 2026 / Volume: 12 Issue: 2

Using Generalizability Theory to Investigate the Reliability of Scores Assigned to Students in English Language Examination in Nigeria

Olufunke Akindahunsi(Obafemi Awolowo University)0000-0002-5041-7088
Eyitayo Rufus Ifedayo Afolabı(Obafemi Awolowa University)
Pages
—
Publication Date
2021-06-30
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The results indicated the g-coefficient to be 0.90 and Φ coefficient as 0.87, which is an indication of high reliability of scores.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:The study investigated the reliability of scores assigned to students in English language in National Examinations Council (NECO).
📊
Key Findings & Data:The population consisted of all the students who sat for NECO Senior School Certificate Examination (SSCE) in 2017 in Nigeria.
💡
Conclusion & Impact:The results indicated the g-coefficient to be 0.90 and Φ coefficient as 0.87, which is an indication of high reliability of scores.
🔬Design:Research Article
📍Scope:2017 Verileri
⚙️Variables:Reliability · Validity · English Language

Abstract

The study investigated the reliability of scores assigned to students in English language in National Examinations Council (NECO). The population consisted of all the students who sat for NECO Senior School Certificate Examination (SSCE) in 2017 in Nigeria. A sample of 311,138 was selected using the proportionate stratified sampling technique. The Optical Marks Record (OMR) sheet containing the responses of the examinees was the instrument for the study. The data was analyzed using lme4 package of R language and environment for statistical computing, factor analysis and Tucker index of factor congruence. The psychometric properties of the data were determined by estimating the generalizability (g) coefficient, phi (Φ) coefficient and construct validity. The results indicated the g-coefficient to be 0.90 and Φ coefficient as 0.87, which is an indication of high reliability of scores. The result also showed that a decrease in the number of the items resulted in a decrease in both g- and phi coefficients in D-study. The construct validity of 0.99 obtained from the result affirms the credibility of the items. Hence, it was concluded that the scores were dependable and generalizable.

🔖

Cite This Article

Akindahunsi, O., Afolabı, E. R. I. (2021). Using Generalizability Theory to Investigate the Reliability of Scores Assigned to Students in English Language Examination in Nigeria. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, Volume: 12 Issue: 2. https://doi.org/10.21031/epod.820989

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (26)

  1. Alkharusi, H. (2012). Generalizability theory: An analysis of variance approach to measurement problems in educational assessment. Journal of Studies in Education, 2(1), 184-196. doi: 10.5296/jse.v2i1.1227
  2. Bates, D., Mächler, M., Bolker, B. M., & Walker, S. C. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1-48. doi: 10.18637/jss.v067.i01
  3. Breithaupt, K. (2011). Medical licensure testing: White paper for the assessment review task force of the medical council of Canada. Retrieved from https://www.mcc.ca/wp-content/uploads/Technical-Reports-Breithaupt-2011.pdf
  4. Brennan, R. L. (2001). Generalizability theory. New York: Springer.
  5. Brennan, R. L. (2003). Coefficients and indices in generalizability theory (CASMA Research Report Number 1). Iowa: Centre for Advanced Studies in Measurement and Assessment.
  6. Crocker, L., & Algina, J. (2008). Introduction to classical and modern test theory. U.S.A: Cengage Learning.
  7. de Gruijter, D. N., & van der Kamp, L. J. Th. (2008). Statistical test theory for the behavioural sciences. New York: Chapman & Hall/CRC.
  8. de Vries, I. M. (2012). An analysis of test construction procedures and score dependability of a paramedic recertification exam (Master’s thesis). Queen’s University Kingston, Ontario, Canada.
  9. Desjardins, C. D., & Bulut, O. (2018). Handbook of educational measurement and psychometrics using R (1st Ed). Parkway, Boca Raton: Chapman and Hall/CRC Press. doi: 10.1201/b20498
  10. Fosnacht, K., & Gonyea, R. M. (2018). The dependability of the updated NSSE: A generalizability study. Research and Practice in Assessment 13, 62-73. Retrieved from https://eric.ed.gov/?id=EJ1203503
  11. Gugiu, M. R., Gugiu, P. C., & Baldus, R. (2012). Utilizing generalizability theory to investigate the reliability of the grades assigned to undergraduate research papers. Journal of Multidisciplinary Evaluation, 8(19), 26-40. Retrieved from https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/362
  12. Johnson, S., & Johnson, R. (2009). Conceptualising and interpreting reliability. Coventy: Ofqual Junker, B. W. (2012). Some aspects of classical reliability theory and classical test theory. Department of Statistics, Carnegie Mellon University, Pittsburgh.
  13. Kamis, O., & Dogan, C. D. (2018). An investigation of reliability coefficients estimated for decision studies in generalizability theory. Journal of Education and Learning, 7(4), 103-113. doi: 10.5539/jel.v7n4p103
  14. Li, M., Shavelson, R. J., Yin, Y., & Wiley, W. (2015). Generalizability theory. In The encyclopedia of clinical psychology (pp. 1322-1340). doi: 10.1002/9781118625392.wbecp352
  15. Lorenzo-Seva, U., & Ten Berge, J. U. M. (2006). Tucker’s congruence coefficient as a meaningful index of factor similarity. Methodology, 2(2), 57-64. doi: 10.1027/1614-2241.2.2.57 Mushquash, C., & O’Connor, B. (2006). SPSS and SAS programs for generalizability theory analyses. Behavioral Research Methods, 38, 542-547. doi: 10.3758/BF03192810
  16. Nalbantoglu-Yilmaz, F. (2017). Reliability of scores obtained from self-, peer-, and teacher-assessments on teaching materials prepared by teacher candidates. Educational Sciences: Theory & Practice, 17(2), 395-409. doi: 10.12738/estp.2017.2.0098
  17. Olusoji, O. A. (2012). Effects of English language on national development. Greener Journal of Social Sciences, 2(4), 134-139. doi: 10.15580/GJSS.2012.4.08291255
  18. Rentz, J. O. (1987). Generalizability theory: A comprehensive method for assessing and improving the dependability of marketing measures. Journal of Marketing Research, 24(1), 19-28. doi: 10.1177/002224378702400102
  19. Shavelson, R. J., & Webb, N. M.1(991). Generalizability theory: A Primer. Newbury Park CA: Sage.
  20. Solano-Flores, G., & Li, M. (2006). The use of generalizability theory in the testing of linguistic minorities. Educational Measurement: Issues and Practice, 25(1), 13-22. doi: 10.1111/j.1745-3992.2006.00048.x
  21. Strube, M. J. (2002). Reliability and generalizability theory. In L.G. Grimm and P. R. Yarnold (Eds.), Reading and understanding more multivariate statistics (pp. 23-66). Washington, DC: American Psychological Association.
  22. Tasdelen-Teker, G., Sahin, M. G., & Baytemir, K. (2016). Using generalizability theory to investigate the reliability of peer assessment. Journal of Human Sciences, 13(3), 5574-5586. Retrieved from https://j-humansciences.com/ojs/index.php/IJHS/article/view/4155
  23. Uzun, N. B., Aktas, M. Asiret, S., & Yorumalz, S. (2018). Using generalizability theory to assess the score reliability of communication skills of dentistry students. Asian Journal of Education and Training, 4(2), 85-90. doi: 10.20448/journal.522.2018.42.85.90
  24. Webb, N. M., Shavelson, R. J., & Haertel, E. H. (2006). 4 reliability coefficients and generalizability theory. Handbook of Statistics, 26, 81-124. doi: 10.1016/S0169-7161(06)26004-8
  25. Yin, Y., & Shavelson, R. J. (2008). Application of generalizability theory to concept map assessment research. Applied Measurement in Education, 21(3), 273-291. doi: 10.1080/08957340802161840
  26. Zainudin, A. (2012). Research methodology and data analysis (5th Ed). Shah Alam: Universiti Teknologi MARA Publication Centre (UiTM Press).