Dergiler / International Journal of Assessment Tools in Education / 2021 / Cilt: 8 - Sayı: 1

Defining Cut Point for Kullback-Leibler Divergence to Detect Answer Copying

Sayfa
156–166
Yayın Tarihi
2022-10-18
DOI
—
🏛️Bu makalenin bibliyografik verileri ve tam metin bağlantısı DergiPark üzerinden sağlanmıştır.
DergiPark Kaydı ↗
✨

Yapay Zeka Hızlı Bakış & Çıkarımlar

AI Insight
⚡ TL;DR(1 Cümlede Hızlı Çıkarım)

When the sample size and the copiers' rate increased, it was observed that the power of the KL method decreased when the cut point with the cost-benefit approach was used.

📌 3 Maddede Bu Makale

🎯
Amaç & Hipotez:Distance learning has become a popular phenomenon across the world during the COVID-19 pandemic.
📊
Temel Bulgular & Veri:Using the cut point obtained, individuals were classified as a copier or not, and the KL method was examined for cases where the determination power of the KL method was 1000, and 3000 sample size, 40 test length, copiers' rate was 0.05 and 0.15, and copying percentage was 0.1, 0.3 and 0.6.
💡
Sonuç & Katkı:When the sample size and the copiers' rate increased, it was observed that the power of the KL method decreased when the cut point with the cost-benefit approach was used.
🔬Tür:Araştırma Makalesi
📍Kapsam:3000 sample
⚙️Değişkenler:Copy detection · Cut point · ROC analysis

Özet

Distance learning has become a popular phenomenon across the world during the COVID-19 pandemic. This led to answer copying behavior among individuals. The cut point of the Kullback-Leibler Divergence (KL) method, one of the copy detecting methods, was calculated using the Youden Index, Cost-Benefit, and Min Score p-value approaches. Using the cut point obtained, individuals were classified as a copier or not, and the KL method was examined for cases where the determination power of the KL method was 1000, and 3000 sample size, 40 test length, copiers' rate was 0.05 and 0.15, and copying percentage was 0.1, 0.3 and 0.6. As a result, when the cut point was obtained with the Min Score p-value approach, one of the cutting methods approaches, it was seen that the power of the KL index to detect copier was high under all conditions. Similarly, under all conditions, it was observed that the second method, in which the detection power of the KL method was high, was the Youden Index approach. When the sample size and the copiers' rate increased, it was observed that the power of the KL method decreased when the cut point with the cost-benefit approach was used.

Abstract

Distance learning has become a popular phenomenon across the world during the COVID-19 pandemic. This led to answer copying behavior among individuals. The cut point of the Kullback-Leibler Divergence (KL) method, one of the copy detecting methods, was calculated using the Youden Index, Cost-Benefit, and Min Score p-value approaches. Using the cut point obtained, individuals were classified as a copier or not, and the KL method was examined for cases where the determination power of the KL method was 1000, and 3000 sample size, 40 test length, copiers' rate was 0.05 and 0.15, and copying percentage was 0.1, 0.3 and 0.6. As a result, when the cut point was obtained with the Min Score p-value approach, one of the cutting methods approaches, it was seen that the power of the KL index to detect copier was high under all conditions. Similarly, under all conditions, it was observed that the second method, in which the detection power of the KL method was high, was the Youden Index approach. When the sample size and the copiers' rate increased, it was observed that the power of the KL method decreased when the cut point with the cost-benefit approach was used.

🔖

Bu Makaleye Atıf Yap

Doğan, C. D., Uçar, A. (2022). Defining Cut Point for Kullback-Leibler Divergence to Detect Answer Copying. International Journal of Assessment Tools in Education, Cilt: 8 - Sayı: 1, 156–166. https://ajindex.com/articles/540078

📄 Tam Metin PDF Okuyucu

📄

PDF görüntüleyici hazırlanıyor...

Kaynakça (12)

  1. Bamber, D. (1975). The area above the ordinal dominance graph and the area below the receiver operating graph. Journal of Mathematical Psychology, 12, 387 415. https://doi.org/10.1016/0022-2496(75)90001-2
  2. Belov, D. I., & Armstrong, R. D. (2010). Automatic detection of answer copying via Kullback–Leibler divergence and K-index. Applied Psychological Measurement, 34, 379–392. https://doi.org/10.1177/0146621610370453
  3. Belov, D. (2011). Detection of Answer Copying Based on the Structure of a High-Stakes Test. Applied Psychological Measurement, 35(7), 495 517. https://doi.org/10.1177/0146621611420705
  4. Belov, D. (2013). Detection of test collusion via Kullback–Leibler divergence. Journal of Educational Measurement,50, 141-163. https://doi.org/10.1111/jedm.12008
  5. Belov, D. (2014a). Detection of Aberrant Answer Changes via Kullback– Leibler Divergence (Report No. RR 14-04).
  6. Law School Admission Council. Belov, D. I. (2014b). Detecting item preknowledge in computerized adaptive testing using information theory and combinatorial optimization. Journal of Computerized Adaptive Testing, 2, 37-58. http://dx.doi.org/10.7333%2Fjcat.v2i0.36
  7. Chalmers, P. (2020). Multidimensional item response model (mirt) [Computer software manual]. https://cran.r project.org/web/packages/mirt/mirt.pdf Chang, H.-H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213 229. https://doi.org/10.1177/014662169602000303
  8. He, Q., Meadows, M., & Black, B. (2018). Statistical techniques for studying anomaly in test results: a review of literature (Report No: Ofqual 6355-5). Office of Qualifications and Examinations Regulation. Hurtz, G., & Weiner, J. (2019). Analysis of test-taker profiles across a suite of statistical indices for detecting the presence and impact of cheating. Journal of Applied Testing Technology, 20(1), 1 15. http://www.jattjournal.com/index.php/atp/article/view/140828
  9. Karabatsos, G. (2003). Comparing the aberrant response detection performance of thirty-six person fit statistics. Applied Measurement in Education, 16(4), 277 298. https://doi.org/10.1207/S15324818AME1604_2
  10. Krzanowski, W., & Hand, D. (2009). ROC curves for continuous data. Chapman and Hall/CRC Press.
  11. Kullback, S., & Leibler, R. (1951). On information and sufficiency. The Annals of Mathematical Statistics, 22(1), 79 86. https://www.jstor.org/stable/2236703
  12. Lindahl, J., & Danell, R. (2016). The information value of early career productivity in mathe