NAEP Writing Achievement Level Setting: Call for Public Comment

Comments Due February 25, 2011

The National Assessment Governing Board (Governing Board) is soliciting public comments and recommendations for improvements to the design proposed for setting achievement levels for the National Assessment of Educational Progress (NAEP) in writing, as described in the study’s Design Document. The proposed Design Document available on this site describes the process that will produce cutscores that represent the lower boundary of each of three NAEP achievement levels: Basic, Proficient, and Advanced. The Governing Board has contracted with WestEd to assist in gathering feedback on the Design Document. Additional information on the Governing Board’s work and NAEP achievement levels can be found at www.nagb.org.

Public and private individuals and organizations are invited to provide written comments and recommendations. Voluntary participation by all interested parties is urged. Following are the review schedule, a description of the kinds of information that the Governing Board is seeking to obtain regarding the Design Document, and information for accessing additional materials that will be useful for this review.

Background

Under Public Law 107-279, the National Assessment Governing Board (NAGB) is authorized to formulate policy guidelines for NAEP. The legislation specifies that the Governing Board is to develop appropriate student achievement levels for each subject and grade tested, as provided in section 303(e). Achievement levels are determined by identifying the knowledge that can be measured and verified objectively using widely accepted professional assessment standards and developing achievement levels that are consistent with relevant widely accepted professional assessment standards and based on the appropriate level of subject matter knowledge for grade levels to be assessed, or on the age of the students.

In preparation for conducting a study to set achievement levels for the new assessment of writing at grades 4, 8, and 12, the Governing Board seeks comment on the draft Design Document intended to guide this study. This is the first wholly computer-based NAEP, and the design calls for this to be the first computerized NAEP achievement levels setting process. Comments are invited, particularly on the computerization of the achievement levels setting process. All responses will be taken into consideration before finalizing the Design Document.

Materials for Review and Comment

The Governing Board is seeking comment from policymakers, teachers, researchers, state and local writing specialists, members of professional writing and teacher organizations, and members of the public.

To assist with review and comment, the following materials are made available here:

1. Design Document: The draft Design Document presents a preliminary design approach to guide all aspects of this study. Download a draft of this document for review purposes.
2. Focus Questions: Download focus questions related to certain aspects of the Design Document that are provided as potential areas of interest that require feedback; these questions are also provided below. Comments will provide valuable feedback that is designed to improve the first computerized achievement levels setting process.


• The objective of this study is to set achievement levels for the 2011 and 2013 NAEP writing assessments. Does the study design as presented in the Design Document seem reasonable for accomplishing this overall study objective?
• What improvements can be made to the design to more fully accomplish the objectives of this study?
• The proposed design calls for the computerization of many aspects of the study. Are there aspects of this computerization that will be particularly effective or ineffective in meeting the objective of this study?
• Is the field trial as described a reasonable method for testing the logistics of the computerized methodology?
• Is the special study as described a reasonable method for comparing performance relative to the achievement levels on the 2007 writing NAEP assessment with performance relative to the achievement levels for the new writing NAEP assessment?

Timeline

It is anticipated that the finalized Design Document will be presented for approval to the Governing Board on March 10. Comments must be received by midnight Pacific Standard Time — February 25, 2011 and sent to:

WestEd, 730 Harrison Street, San Francisco, CA, 94107, Attention: Jennae Bulat: Public Comment, FAX: (415) 615-3200, E-mail: JBulat@Wested.org.

For Further Information Contact:

Jennae Bulat, WestEd, 730 Harrison Street, San Francisco, CA, 94107, Telephone: (415) 615-3260, FAX: (415) 615-3200, E-mail: JBulat@Wested.org.

Electronic Access to Documents: The documents available on the web page are in Adobe Portable Document Format (PDF). To use PDF you must have Adobe Acrobat Reader, which is available free at get.adobe.com/reader.