Measuring Resilience and Youth Development: The Psychometric Properties of the Healthy Kids Survey

Methodologies: Survey Research

Researchers with WestEd's Regional Educational Laboratory West (REL West) studied the psychometric properties of the resilience and youth development module, a key component of the agency's Healthy Kids Survey (HKS). HKS is a comprehensive student self-report tool for monitoring the school environment and student health risks.

The goal of this study was to improve resilience assessment and research so that educators can shape the school environment to promote academic resilience. The research study resulted in a report, prepared by REL West, which summarizes the findings. The report focuses on one module of the survey, the resilience and youth development module (RYDM), which assesses environmental and internal assets associated with positive youth development and school success. Environmental assets refer to meaningful and prosocial bonding to community, school, family, and peers. Internal assets are personal resilience traits, such as self-efficacy and problem-solving skills.

The report describes the results of the study analysis, provides recommendations on the proper use of the HKS, and suggests modifications to the module.

Methodology

The purpose of the study was to describe the psychometric properties of the RYDM secondary school and elementary school instruments. Specifically, REL West researchers examined:

• the dimensionality of scales via exploratory and confirmatory factor analysis models;
• measurement equivalence across demographic subgroups by estimating confirmatory factor analysis models with covariates (i.e., multiple indicator, multiple cause structural equation models);
• scale reliability by estimating internal consistency and test-retest reliability coefficients; and
• construct validity by examining the relationship of scales to other theoretically related constructs and mean differences across demographic subgroups.

The analyses are based on HKS data processed for school districts by WestEd's Health and Human Development Program. These statewide data were supplemented by two sets of HKS data originally collected for other purposes. Data collected in 2006 from a large urban school district in Southern California were used to describe the temporal stability of the derived scales (test-retest reliability). Data collected in 2004/05 from students in a large county in Southern California were used to examine the relationship between the RYDM constructs and standardized test scores.

Key Findings

The secondary school RYDM module is a relatively short instrument suitable for widespread administration. REL West's analyses suggest that it provides comprehensive and balanced coverage of both environmental (eight dimensions) and internal (four dimensions) resilience assets; its subscales exhibit good internal consistency and are associated with student risk factors in expected ways; and, if certain items are omitted, it demonstrates measurement equivalence across racial/ethnic groups, males and females, and grades.

The results are generally consistent with current usage of the instrument as an epidemiological tool, as well as with the conceptual foundation underlying it. However, the secondary school resilience and youth development scales exhibit fairly low levels of test-retest reliability, which suggests that the instrument is not well-suited for examining student-level changes over time. The instrument is not designed to be used for local program evaluations aimed at examining individual differences across students, and REL West's analyses suggest that it should not be used for such purposes. Moreover, the results suggest that two (cooperation and goals/aspirations) of the six personal strengths (internal assets) that the secondary school RYDM was designed to measure could not be assessed validly. In addition, several of the measures would benefit if additional items were developed and included in derived scales to increase domain coverage.

The elementary school resilience instrument was designed to assess seven environmental assets and three internal assets. Research on how to measure protective factors and resilience traits among elementary school students is lacking, and REL West's measurement analyses of the elementary school RYDM suggest that further work is needed in this area. The analyses suggest that the elementary school RYDM appears to be able to reliably assess only two measures of environmental assets and one internal asset measure. Most of the scales measured by the elementary school instrument have relatively poor psychometric properties. REL West researchers, therefore, recommend that the elementary school instrument undergo considerable modification to make it more suitable for general research.

Contact Information

Thomas Hanson
562.799.5170
thanson@wested.org