SavedAdd your journal

Journals / International Journal of Assessment Tools in Education / 2016 / Cilt: 3 - Sayı: 2

The Influence of Item Formats when Locating a Student on a Learning Progression in Science

Jing CHEN
Charles W. ANDERSON
Amelia Wenk GOTWALS
Mark D. RECKASE
Pages
101–122
Publication Date
2022-10-18
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

Our results suggest that estimates of students’ learning progression level based on OMC and MTF responses are moderately predictive of their level based on CR responses.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Learning progressions are used to describe how students’ understanding of a topic progresses over time.
📊
Key Findings & Data:The item formats investigated were Constructed Response (CR) items and two types of two-tier items: (1) Ordered Multiple-Choice (OMC) followed by CR items and (2) Multiple True or False (MTF) followed by CR items.
💡
Conclusion & Impact:Our results suggest that estimates of students’ learning progression level based on OMC and MTF responses are moderately predictive of their level based on CR responses.
🔬Design:Research Article
📍Scope:Comprehensive Sample
⚙️Variables:Item Format · Item Response Theory (IRT) · Learning Progression

Abstract

Learning progressions are used to describe how students’ understanding of a topic progresses over time. This study evaluates the effectiveness of different item formats for placing students into levels along a learning progression for carbon cycling. The item formats investigated were Constructed Response (CR) items and two types of two-tier items: (1) Ordered Multiple-Choice (OMC) followed by CR items and (2) Multiple True or False (MTF) followed by CR items. Our results suggest that estimates of students’ learning progression level based on OMC and MTF responses are moderately predictive of their level based on CR responses. With few exceptions, CR items were effective for differentiating students among learning progression levels. Based on the results, we discuss how to design and best use items in each format to more accurately measure students’ level along learning progressions in science.

Özet

Learning progressions are used to describe how students’ understanding of a topic progresses over time. This study evaluates the effectiveness of different item formats for placing students into levels along a learning progression for carbon cycling. The item formats investigated were Constructed Response (CR) items and two types of two-tier items: (1) Ordered Multiple-Choice (OMC) followed by CR items and (2) Multiple True or False (MTF) followed by CR items. Our results suggest that estimates of students’ learning progression level based on OMC and MTF responses are moderately predictive of their level based on CR responses. With few exceptions, CR items were effective for differentiating students among learning progression levels. Based on the results, we discuss how to design and best use items in each format to more accurately measure students’ level along learning progressions in science.

🔖

Cite This Article

CHEN, J., ANDERSON, C. W., GOTWALS, A. W., RECKASE, M. D. (2022). The Influence of Item Formats when Locating a Student on a Learning Progression in Science. International Journal of Assessment Tools in Education, Cilt: 3 - Sayı: 2, 101–122. https://ajindex.com/articles/540263

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (7)

  1. Adams, R.J., Wilson, M., & Wang, W-C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1-23.
  2. Alonzo, A.C., & Gotwals, A. G. (2012). Learning progressions in science. Rotterdam, The Netherlands: Sense Publishers.
  3. Alonzo, A.C., & Steedle, J. T. (2008). Developing and assessing a force and motion learning progression. Published online in Wiley InterScience. Anderson, C.W., Alonzo, A. C., Smith, C., & Wilson, M. (2007, August). NAEP pilot learning progression framework. Report to the National Assessment Governing Board. Angoff, W.H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508-600).
  4. Washington, DC: American Council on Education. Berlak, H. (1992). The need for a new science of assessment. In H. Berlak et al. (Eds.) Toward a new science of educational testing and assessment (pp. 1-22). Albany: State University of New York Press.
  5. Briggs, D.C. & Alonzo, A.C. (2012). The psychometric modelling of ordered multiple-choice item responses for diagnostic assessment with a learning progression. In A.C. Alonzo & A.W. Gotwals (eds). Learning progressions in science: Current challenges and future directions. Rotterdam, The Netherlands: Sense Publishers.
  6. Briggs, D.C., Alonzo, A.C., Schwab, C., & Wilson, M. (2006). Diagnostic assessment with ordered multiple-choice items. Educational Assessment, 11, 33 – 63. Catley, K., Lehrer R., & Reiser, B. (2004). Tracing a prospective learning progression for developing understanding of evolution, Paper Commissioned by the National Academies Committee on Test Design for K–12 Science Achievement, Washington, DC: National Academy of Science, 67. Chen, J. & Anderson, C.W. (2015). Comparing American and Chinese K-12 students’ learning progression on carbon cycling in socio-ecological systems. Science Education International. 27(4), 439-462.
  7. Corcoran, T., Mosher, F. A., & Rogat, A. (2009, May). Learning progressions in science: An evidence based approach to reform (CPRE Research Report #RR-63). Philadelphia, PA: Consortium for Policy Research in Education. Doherty, J., Draney, K., Shin, H., Kim, J., & Anderson, C. W. (2015). Validation of a learning progression-based monitoring assessment. Manuscript submitted for publication. Downing, S. M., & Yudkowsky, R. (2009). Assessment in Health Professions Education. New York, NY Routledge. Dunham, M. L. (2007) An investigation of the multiple true-false item for nursi