 Online Learning Update Ray Schroeder, editor, COLRS - Online@Illinois SpringfieldOnline Learning Update - a blog published daily since 2001 by Ray Schroeder, Professor Emeritus, University of Illinois at Springfield |
|
|
Online Learning News and Research
Online Learning Update HOME Times and Dates Coordinated Universal Time
Search Online Learning Blog Archives
|
Saturday, May 05, 2007
The Effect of Using Item Parameters Calibrated from Paper Administrations in Computer Adaptive Test Administrations - Mary Pommerich, JTLA
Computer administered tests are becoming increasingly prevalent as computer technology becomes more readily available on a large scale. For testing programs that utilize both computer and paper administrations, mode effects are problematic in that they can result in examinee scores that are artificially inflated or deflated. As such, researchers have engaged in extensive studies of whether scores differ across paper and computer presentations of the same tests. The research generally seems to indicate that the more complicated it is to present or take a test on computer, the greater the possibility of mode effects. In a computer adaptive test, mode effects may be a particular concern if items are calibrated using item responses obtained from one administration mode (i.e., paper), and those parameters are then used operationally in a different administration mode (i.e., computer). This paper studies the suitability of using parameters calibrated from a paper administration for item selection and scoring in a computer adaptive administration, for two tests with lengthy passages that required navigation in the computer administration. The results showed that the use of paper calibrated parameters versus computer calibrated parameters in computer adaptive administrations had small to moderate effects on the reliability of examinee scores, at fairly short test lengths. This effect was generally diminished for longer test lengths. However, the results suggest that in some cases, some loss in reliability might be inevitable if paper-calibrated parameters are used in computer adaptive administrations.
Comments:
Post a Comment
Online Learning News Blog Archives OTEL - Ray's Home Page - Notebook - UIS Online - U of I Online - UIS Home Fair Use
|
