Cultural Validity in Assessment Project
Assessing the Cultural Validity of Science and Mathematics Assessments

Director: Sharon Nelson-Barber    (Director, Culture and Language in Education Research) Contact: Guillermo Solano-Flores Tel: 650.381.6412 Email: wsolano@wested.org

The Cultural Validity in Assessment (CVA) project investigates how cultural background influences the ways in which students interpret science and mathematics assessment items and the cognitive activities they use in completing those items. To carry out our investigation, we administered science and mathematics test items to a sample of students from ten cultural groups and geographical areas in the United States. The test items have been used in the past in large-scale testing (specifically, the National Assessment of Educational Progress or NAEP) and are deemed psychometrically sound. Also, we worked individually with a sub-sample of students for each cultural group and asked them to "think aloud" as they engaged in responding to the items. After completing each item, we asked students to explain how they interpreted it and how they related its content to their personal experience. We are in the process of examining the mean score differences between cultural groups for each item, the inferred cognitive activity of the students as they take the tests, and the students' socio-cultural activity (participation, context, and system of values, beliefs, and epistemologies). Based on our analysis of the data, we will determine: (1) whether students from different cultural groups exhibit different patterns by which they understand the science and mathematics exercises; (2) how culture influences the inferred cognitive activity elicited by those exercises; and (3) whether those differences can account for performance score differences among cultural groups. Our ultimate goal is to contribute to equitable testing in science and mathematics by offering a new perspective for addressing cultural diversity in testing. Toward this end, we intend to determine whether procedures for test development and evaluation should pay more attention to culture and whether the notion of cultural validity should be considered in assessment development and testing practices. The project has created a website and a newsletter, Chrysalis, that is available online. Chrysalis provides teachers and schools with information on the progress of the project, along with information intended to be useful for practitioners of science and mathematics education in multicultural contexts.

Who's involved with this Project? Rebeca Diaz (Research Associate) Jo Ann Izu (Senior Research Associate) Sharon Nelson-Barber (Director, Culture and Language in Education Research) Ursula Sexton (Senior Research Associate)

What kind of Resources are available? R&D Alert® Vol. 3, No. 3 Contents Creating a Test That Is Also a Science Learning Tool From the CEO: Assessment for Education Improvement Pooling Resources and Expertise: The Arizona Assessment Collaborative Assessing Students for What They Know, Not Where They Come From Assuring Quality Language Instruction Implementing Standards Addressing (and Assessing) Precursors to Academic Failure Including Students with Disabilities in Assessment Spreading the Word on Continuous Assessment WestEd's Assessment Resources Single copies are available free of charge. All current and past issues of R&D Alert are available online. Testing English Language Learners: A Sampler of Student Responses to Science and Mathematics Test Items State and district officials are working to obtain valid measures of academic achievement in English learners and thus insure their inclusion in programs for the assessment of content knowledge. This sampler is a resource for these efforts. It analyzes the responses of elementary school native speakers of Spanish, Haitian-Creole, and Chinese who were given open-ended science and mathematics items in both English and their native languages. This comparison of student responses across languages is intended to enhance educators’ understanding of English learners and the interpretation of their responses to test items. The document also includes a series of exercises designed to promote the use of the sampler as a tool for professional development in the area of assessment.