SavedAdd your journal

Journals / Hacettepe Üniversitesi Eğitim Fakültesi Dergisi / 2013 / Cilt: 2013 - Sayı: 44

Comparing student evaluation formats in terms of halo effect, leniency effect and user reactions

Öğretim elemanı değerlendirmesinde kullanılan formatların hale etkisi, cömertlik etkisi ve kullanıcı tepkileri açısından karşılaştırılması

Afife Başak OK
Reyhan BİLGİÇ
H Canan SÜMER
Pages
237–248
Publication Date
2022-10-19
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

Considering the differential resources required for the development of a behaviour-based rating scale, the present results indicated the GRS format as being preferable to the BARS format.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:The purpose of this study was to compare two different rating scales to be used in student evaluation of instructors in terms of psychometric properties, mainly halo and leniency effects, and user reactions.
📊
Key Findings & Data:A behaviourally anchored rating scale (BARS) and a graphic rating scale (GRS) were developed and administered to a total sample of 270 college students (NBARS = 126; NGRS = 144) rating their real-life instructors.
💡
Conclusion & Impact:Considering the differential resources required for the development of a behaviour-based rating scale, the present results indicated the GRS format as being preferable to the BARS format.
🔬Design:Research Article
📍Scope:Comprehensive Sample
⚙️Variables:Domain Parameters

Abstract

The purpose of this study was to compare two different rating scales to be used in student evaluation of instructors in terms of psychometric properties, mainly halo and leniency effects, and user reactions. A behaviourally anchored rating scale (BARS) and a graphic rating scale (GRS) were developed and administered to a total sample of 270 college students (NBARS = 126; NGRS = 144) rating their real-life instructors. Although neither format had a clear psychometric advantage over the other, the GRS format was found to be slightly better than the BARS format in terms of resistance to halo. Furthermore, the BARS format received less favourable user reactions than did the GRS format. Considering the differential resources required for the development of a behaviour-based rating scale, the present results indicated the GRS format as being preferable to the BARS format. The implications of the findings are discussed along with suggestions for future research.

Özet

Bu çalışmanın amacı öğrencilerin öğretim elemanı performansını değerlendirmede kullanılabilecek iki farklı değerlendirme formatını psikometrik özellikleri (hale ve cömertlik/aşırı olumluluk etkileri) ve kullanıcı tepkileri açısından karşılaştırmaktır. Bu amaçla geliştirilen davranış odaklı değerlendirme ölçeği (DODÖ) ve grafik değerlendirme ölçeği (GDÖ) toplam 270 üniversite öğrencisine (NBARS = 126; NGRS = 144) uygulanmış ve öğrencilerden gerçek hayattaki öğretim elemanlarının ders sırasındaki performansını değerlendirmeleri istenmiştir. Formatlardan hiçbiri psikometrik açıdan diğerinden belirgin bir şekilde üstün bulunmamakla birlikte GDÖnün hale etkisine dayanıklılık açısından DODÖden biraz daha iyi olduğu bulunmuştur. Ayrıca, GDÖ ile karşılaştırıldığında DODÖ kullanıcılardan daha az olumlu tepkiler almıştır. Davranış odaklı bir değerlendirme ölçeği geliştirmek için gereken farklı kaynaklar düşünüldüğünde GDÖnün DODÖye tercih edilebileceği yönünde bulgular elde edilmiştir. Elde edilen bulguların sonuçları geleceğe yönelik önerilerle birlikte tartışılmaktadır.

🔖

Cite This Article

OK, A. B., BİLGİÇ, R., SÜMER, H. C. (2022). Comparing student evaluation formats in terms of halo effect, leniency effect and user reactions. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, Cilt: 2013 - Sayı: 44, 237–248. https://ajindex.com/articles/531955

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (14)

  1. Abrami, P. C., & d’Apollonia, S. (1990). The dimensionality of ratings and their use in personnel decisions. In M. Theall & J. Franklin (Eds.), Student ratings of instruction: Issues for improving practice (pp. 97-111). San Francisco: Jossey-Bass. Bernardin, H. J. (2005). Behaviorally anchored rating scales. In Blackwell Encyclopedic Dictionary of Human Resource Management (pp. 22-23).
  2. Oxford, England: Blackwell Publishing. Borman, W. C., & Dunnette, M. D. (1975). Behavior-based versus trait-oriented performance ratings: An empirical study. Journal of Applied Psychology, 60(5), 561-565.
  3. Cooper, W. (1981). Ubiquitous halo. Psychological Bulletin, 90, 218-244. D’Apollonia, S. & Abrami, P. C. (1997). Navigating student ratings of instruction. American Psychologist, 52(11), 1198-1208.
  4. Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51, 327-358.
  5. Heckert, T. M., Latier, A., Ringwald, A., & Silvey, B. (2006). Relation of course, instructor, and student characteristics to dimensions of student ratings of teaching effectiveness. College Student Journal, 40(1), 195-203.
  6. Jelley, R. B. & Goffin, R. D. (2001). Can performance-feedback accuracy be improved? Effects of rater priming and rating-scale format on rating accuracy. Journal of Applied Psychology, 86(1), 134-144.
  7. Kingstrom, P. O., & Bass, A. R. (1981). A critical analysis of studies comparing behaviorally anchored rating scales (BARS) and other rating formats. Personnel Psychology, 34, 263-289.
  8. Kline, T. J. B., & Sulsky, L. M. (2009). Measurement and assessment issues in performance appraisal. Canadian Psychology, 50(3), 161-171.
  9. Krzystofiak, F., Cardy, R., & Newman, J. (1988). Implicit personality and performance appraisal: The influence of trait inferences on evaluations of behavior. Journal of Applied Psychology, 73(3), 515-521.
  10. Landy, F. J., & Farr, J. L. (1980). Performance rating. Psychological Bulletin, 87(1), 72-107.
  11. Landy, F. J., & Farr, J. L. (1983). The measurement of work performance: Methods, theory, and applications. New York: Academic Press.
  12. Lowman, J. (1994). Professors as performers and motivators. College Teaching, 42(4), 137-141.
  13. Madesen, C. K. & Napoles, J. (2006). A 30-year follow-up study of perceptions of students’ ratings of former instructors. UPDATE: Applications of Research in Music Education, 24(2), 45-53.
  14. Marsh, H. W. & Roche, L. A. (1997). Making students’ evaluations of teaching effectiveness effective: The critical issues of validity, bias, and uti