Dergiler / Kuram ve Uygulamada Eğitim Bilimleri / 2016 / Cilt: 16 - Sayı: 4

Investigation of Coefficient of Individual Agreement in Terms of Sample Size, Random and Monotone Missing Ratio, and Number of Repeated Measures

Sayfa
1381–1395
DOI
—
🏛️Bu makalenin bibliyografik verileri ve tam metin bağlantısı DergiPark üzerinden sağlanmıştır.
DergiPark Kaydı ↗
✨

Yapay Zeka Hızlı Bakış & Çıkarımlar

AI Insight
⚡ TL;DR(1 Cümlede Hızlı Çıkarım)

Therefore, it is suggested that data obtained from studies be interpreted considering missing value ratio.

📌 3 Maddede Bu Makale

🎯
Amaç & Hipotez:Studies based on longitudinal data focus on the change and development of the situation being investigated and allow for examining cases regarding education, individual development, cultural change, and socioeconomic improvement in time.
📊
Temel Bulgular & Veri:This study presents the coefficient of individual agreement (CIA) developed for studies in which more than one rater are needed to provide repeated scoring and examines the applications of this agreement coefficient in relation to different sample size, missing value and number of repeated measures.
💡
Sonuç & Katkı:Therefore, it is suggested that data obtained from studies be interpreted considering missing value ratio.
🔬Tür:Araştırma Makalesi
📍Kapsam:Genişletilmiş Akademik Örneklem
⚙️Değişkenler:Alan Parametreleri

Abstract

Studies based on longitudinal data focus on the change and development of the situation being investigated and allow for examining cases regarding education, individual development, cultural change, and socioeconomic improvement in time. However, as these studies require taking repeated measures in different time periods, they may include various error sources that are exclusive of the investigated variable and that change over time. In Addition, the period of observation is critical in these studies, and observant (variable) attrition or the inability to obtain measurements for various reasons (death, moving away, resignation etc.) may be observed. This study presents the coefficient of individual agreement (CIA) developed for studies in which more than one rater are needed to provide repeated scoring and examines the applications of this agreement coefficient in relation to different sample size, missing value and number of repeated measures. In this context, trial scores were generated based on different sample sizes, missing value ratios, and the number of repeated measures provided by two raters, and the expected agreement between raters was simulated as very high. Related data were replicated 1000 times. Data analysis included the mean and standard deviation of agreement ratios calculated for each combination. According to the results, independently from sample size and number of retests, the CIA takes much lower values than the expected value when random missing value rates reaches 20%. Similar conclusions can be drawn for the existence of monotone missing values except where the sample size is 30 and the number of retests are two and three. It was observed that the CIA was higher than the expected value for the data consisting of two repeated evaluations of the raters with 5% or 10% missing values when the sample size was 30. In general, the agreement coefficient did not exceed the expected level in the situation of three and four replications of measurements. As a result, it can be concluded that an increase in the replication number of measurement decreased the obtained agreement values. The obtained data showed that missing value ratio caused differentiations in the CIA. As a result, increases in missing value ratio cause inconsistencies in CIA. Therefore, it is suggested that data obtained from studies be interpreted considering missing value ratio.

🔖

Bu Makaleye Atıf Yap

TEMEL, G. O., ERDOĞAN, S., SELVİ, H., ERSÖZ, İ. K. (2016). Investigation of Coefficient of Individual Agreement in Terms of Sample Size, Random and Monotone Missing Ratio, and Number of Repeated Measures. Kuram ve Uygulamada Eğitim Bilimleri, Cilt: 16 - Sayı: 4, 1381–1395. https://ajindex.com/articles/346040

Kaynakça (14)

  1. Acock, A. A. (2005). Working with missing values. Journal of Marriage and Family, 65, 1012-1028.
  2. Allison, P. D. (2009). Missing data. Quantitative methods in psychology. London, UK: Sage. Aiken L., (2000). Psychological testing and assessment. Boston: Allyn & Bacon. Bernhard, J., Cella, D. F., Coates, A. S., Fallowfield, L., Ganz, P. A., Mainpur, C. M.,. Hürny, C. (1998). Missing quality of life data in cancer clinical trials: Serious problems and challenges. Statistics in Medicine, 17, 517-532.
  3. Bennett, D. A. (2001). How can I deal with missing data in my study? Australian and New Zealand Journal of Public Health, 25, 464-469.
  4. Carletta, J. (1996). Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22(2), 249-254.
  5. Cohen, J. (1960). A coefficient of agreement for nominal scales. Education and Psychological Measurement, 20, 37-46.
  6. Cohen, J. (1990). A coefficient of agreement for nominal scales. Educational and psychological Measurement, 20(1), 37-46.
  7. Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. California, CA: Holt, Rinehart and Winston Inc. Çokluk, Ö., & Kayri, M. (2011). Kayıp değerlere yaklaşık değer atama yöntemlerinin ölçme araçlarının geçerlik ve güvenirliği üzerindeki etkisi [The effects of methods of ımputation for missing values on the validity and reliability of scales]. Educational Sciences: Theory & Practice, 11, 289-309.
  8. Delice, A. (2010). Nicel araştırmalarda örneklem sorunu [The sampling ıssues in quantitative research]. Educational Sciences: Theory & Practice, 10, 1969-2018.
  9. Demir, E., & Parlak, B. (2012). Türkiye'de eğitim araştırmalarında kayıp veri sorunu [Missing value issue at educational sciences in Turkey]. Journal of Measurement and Evaluation in Education and Psychology, 3, 230-241.
  10. Enders, C. K. (2010). Applied missing data analysis. New York, NY: The Guilford Press.
  11. Erdoğan, E. (2011). Bilim ve metafizik üzerine tarihsel bir soruşturma [A historical investigation on science and metaphysics] İstanbul, Turkey: Arkeoloji ve Sanat Yayınları.
  12. Erkuş, A. (2006). Sınıf öğretmenleri için ölçme ve değerlendirme: Kavramlar ve uygulamalar [Measurement and evaluation for primary school teacher: Concepts and applications]. Ankara, Turkey: Ekinoks Yayınları.
  13. Gao, J., Pan, Y., & Haber, M. (2011). Assessment of observer agreement for matched repeated binary measurements. Computational Statistics and Data Analysis, 56, 1052-1060.
  14. Güler, N., & Gelbal, S. (2010). Açık