KaydedilenlerDerginizi ekleyin

Dergiler / International Journal of Assessment Tools in Education / 2022 / Cilt: 9 - Sayı: 1

Standard Setting in Academic Writing Assessment through Objective Standard Setting Method

Fatıma Nur FİŞNE
İsmail KARAKAYA
Mehmet ŞATA
Sayfa
80–97
Yayın Tarihi
2022-10-18
DOI
—
🏛️Bu makalenin bibliyografik verileri ve tam metin bağlantısı DergiPark üzerinden sağlanmıştır.
DergiPark Kaydı ↗
✨

Yapay Zeka Hızlı Bakış & Çıkarımlar

AI Insight
⚡ TL;DR(1 Cümlede Hızlı Çıkarım)

That is, OSS method may be practically used to help the stakeholders make objective judgments on the examinees’ target performance.

📌 3 Maddede Bu Makale

🎯
Amaç & Hipotez:Performance standards have important consequences for all the stakeholders in the assessment of L2 academic writing.
📊
Temel Bulgular & Veri:These standards not only describe the level of writing performance but also provide a basis for making evaluative decisions on the academic writing.
💡
Sonuç & Katkı:That is, OSS method may be practically used to help the stakeholders make objective judgments on the examinees’ target performance.
🔬Tür:Modelleme & Sayısal Simülasyon
📍Kapsam:Genişletilmiş Akademik Örneklem
⚙️Değişkenler:L2 academic writing assessment · Many-facet Rasch measurement model · Standard setting

Özet

Performance standards have important consequences for all the stakeholders in the assessment of L2 academic writing. These standards not only describe the level of writing performance but also provide a basis for making evaluative decisions on the academic writing. Such a high-stakes role of the performance standards requires the enhancement of objectivity in standard setting procedure. Accordingly, this study aims to shed light upon the usefulness of Objective Standard Setting (OSS) method in specifying the levels of proficiency in L2 academic writing. On the basis of the descriptive research design, the sample of this research includes the examinees and raters who were student teachers at the university level. Essay task and analytical writing scoring rubric were employed as the data collection tools. In data analysis, OSS method and two-step cluster analysis were used. The analysis results of OSS method based on many-facet Rasch measurement model (MFRM) outline the distribution of the criteria into the levels of proficiency. Also, the main findings in OSS method were validated with two-step cluster analysis. That is, OSS method may be practically used to help the stakeholders make objective judgments on the examinees’ target performance.

Abstract

Performance standards have important consequences for all the stakeholders in the assessment of L2 academic writing. These standards not only describe the level of writing performance but also provide a basis for making evaluative decisions on the academic writing. Such a high-stakes role of the performance standards requires the enhancement of objectivity in standard setting procedure. Accordingly, this study aims to shed light upon the usefulness of Objective Standard Setting (OSS) method in specifying the levels of proficiency in L2 academic writing. On the basis of the descriptive research design, the sample of this research includes the examinees and raters who were student teachers at the university level. Essay task and analytical writing scoring rubric were employed as the data collection tools. In data analysis, OSS method and two-step cluster analysis were used. The analysis results of OSS method based on many-facet Rasch measurement model (MFRM) outline the distribution of the criteria into the levels of proficiency. Also, the main findings in OSS method were validated with two-step cluster analysis. That is, OSS method may be practically used to help the stakeholders make objective judgments on the examinees’ target performance.

🔖

Bu Makaleye Atıf Yap

FİŞNE, F. N., KARAKAYA, İ., ŞATA, M. (2022). Standard Setting in Academic Writing Assessment through Objective Standard Setting Method. International Journal of Assessment Tools in Education, Cilt: 9 - Sayı: 1, 80–97. https://ajindex.com/articles/540044

📄 Tam Metin PDF Okuyucu

📄

PDF görüntüleyici hazırlanıyor...

Kaynakça (9)

  1. Bejar, I.I. (2008). Standard setting: What is it? Why is it important? R&D Connections, 7, 1-6.
  2. Best, J.W., & Khan, J.V. (2006). Research in Education (10th Edition). Pearson. Bichi, A.A., Talib, R., Embong, R., Mohamed, H. B., Ismail, M. S., & Ibrahim, A. (2019). Rasch-based objective standard setting for university placement test. Eurasian Journal of Educational Research, 19(84), 57-70. https://doi.org/10.14689/ejer.2019.84.3
  3. Chen, W.H., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22(3), 265-289. https://doi.org/10.3102/10769986022003265
  4. Cizek, G.J. (1993). Reconsidering standards and criteria. Journal of Educational Measurement, 30(2), 93-106. https://doi.org/10.1111/j.1745-3984.1993.tb01068.x
  5. Cizek, G.J. (Ed.). (2012). An introduction to contemporary standard setting: concepts, characteristics, and concepts. In Setting performance standards: Concepts, methods, and perspectives. Mahwah, NJ: Lawrence Erlbaum Associates. Council of Europe [CoE]. (2001). Common European framework of reference for languages: Learning, teaching, assessment. Cambridge, England: Cambridge University Press.
  6. Cokluk, O., Sekercioglu, G., & Buyukozturk, S. (2012). Sosyal bilimler icin cok degiskenli istatistik: SPSS ve LISREL uygulamalari (2nd edition) [Multivariate statistics for social sciences: SPSS and LISREL applications], Pegem Akademi. Davies, A., Brown, A., Elder, C., Hill, K., Lumley, T., & McNamara, T. (1999). Studies in language testing 7: Dictionary of language testing. Cambridge University Press.
  7. Davis-Becker, S.L., Buckendahl, C.W., & Gerrow, J. (2011). Evaluating the bookmark standard setting method: The impact of random item ordering. International Journal of Testing, 11(1), 24-37. https://doi.org/10.1080/15305058.2010.501536
  8. Elder, C., Barkhuizen, G., Knoch, U., & Von Randow, J. (2007). Evaluating rater responses to an online training program for L2 writing assessment. Language Testing, 24(1), 37-64. https://doi.org/10.1177/0265532207071511
  9. Erkus, A., Sunbul, O., Omur-Sunbul, S., Yormaz, S., & Asiret, S. (2017). Psikolojide olcme ve olcek gelistirme-II (1st edition) [Measurement in psychology and scale development-II], Pegem Akademi. Fleckenstein, J., Keller, S., Krüger, M., Tannenbaum, R.J., & Köller, O. (2020). Linking TOEFL iBT® writing rubrics to CEFR levels: Cut scores and validity evidence from a standard setting study. Assessing Writing, 43, 1 15. https://doi.org