Published by the National
Center on Educational Outcomes
Number 7 / November, 2002
Summary of Teleconference on Building Tests to Support Instruction and Accountability for All Students
On
Commission
Recommendations
Jim Popham initiated
the discussion by providing a rationale for thinking about assessments that not only meet
accountability requirements, but that also inform instruction. He noted the work of the
Commission on Instructionally Supportive Assessment, which produced a set of nine
requirements for these types of assessments (presented in Building Tests to Support Instruction and
Accountability: A Guide for Policymakers), and complimented the five associations that
supported the Commission’s independent work (AASA, NAESP, NASSP, NEA, NMSA).
Jim spent time on
each of the nine requirements, elaborating the importance, benefits, and challenges of
each requirement. Considerable time was spent on the first three requirements, which are
viewed as essential first steps toward building tests that can support both instruction
and accountability. These requirements demand attention to identifying and describing in
detail those high priority standards (the “big boppers” or “large lumps”)
to be assessed through state tests. Lower priority standards are to be measured through
classroom assessments and monitored by the state, so that unintended narrowing of the
curriculum does not occur. Following elaboration of the additional requirements for these
assessments, including that they be well designed and appropriate for a broad range of
students, with accommodations and alternate methods of assessment, Jim turned the
presentation over to
Report
from
As
This challenge to
provide states with more guidance has been taken seriously by Jim and other Commission
members. Work has already started on identifying specific examples of priority (“big
bopper”) standards that states can use to model their own efforts. Several states are
convening to generate more discussion about examples, and test publishing companies are
being asked to join in the discussion about building tests that support instruction and
accountability.
Discussion
The federal
representatives provided general comments on the potential of the approach described by
Jim Popham for building test to support instruction and accountability. In addition,
several states asked specific questions of Jim Popham and the
Next
Teleconference
Be sure to reserve
Three NCEO
publications highlight approaches that can be used to set achievement standards for
alternate assessments. These approaches are similar to those used with general
assessments. The first report, authored by Edward Roeber (Setting Standards on Alternate Assessments, Synthesis
Report 42) points out that all types of alternate assessments need to be scored and
assigned proficiency levels. Ed summarizes a variety of approaches used in general
assessment (including reasoned judgment, contrasting groups, modified Angoff, bookmarking
or item mapping, body of work, and judgmental policy capturing), but cautions that both
technical and practical considerations must be weighed while choosing an approach.
The second report, by
Barbara Olson, Ronald Mead, and David Payne (A
Report of a Standard Setting Method for Alternate Assessments for Students with
Significant Disabilities, Synthesis Report 47),
describes a body of work standard setting approach used in one state for a standards-based
portfolio alternate assessment. The authors identify time and resource constraints in this
method, and emphasize the importance of range-finding and pinpointing phases for the body
of work approach. Through a detailed description of the rationale, design, and process
they also suggest that with careful planning, standards can be set for alternate portfolio
assessments just as they can for any other assessment.
In the third report,
Dan Wiener describes the theoretical debates and decisions that formed the basis for
another state’s portfolio method of alternate assessment, decisions that also shaped
the method used to set standards for the assessment (Massachusetts:
One State’s Approach to Setting Performance Levels on the Alternate Assessment, Synthesis Report 48). One early policy decision
that affected standard setting was that the alternate assessment in
These three reports
can be found at:
Roeber, E. (2002). Setting standards on alternate assessments (Synthesis Report 42). http://cehd.umn.edu/NCEO/OnlinePubs/Synthesis42.html
Olson, B., Mead, R.,
& Payne, D. (2002). A report of a standard
setting method for alternate assessments for students with significant disabilities(Synthesis
Report 47). http://cehd.umn.edu/NCEO/OnlinePubs/Synthesis47.html
Wiener, D. (2002).