Part II
Standards Research and Development Methodology

• Background Research
• Standards Development
• Standards Review and Validation
• Pilot Implementation Study

Background Research

Countless professional organizations and health care agencies have defined those competencies required in their respective health care occupations. To provide a strong starting point for the standards development process, FWL staff reviewed this previous work on competencies, as well as several strands of related research (e.g., in allied health professions, organizational theory, consensus group methodology, and work redesign). Data were then collected from a sample of existing analyses that identified the specific tasks health professionals perform as part of their duties. Sixty occupations across 10 different data sets were carefully assessed. Competencies established by professional associations were examined as well to assure that the NHCSSP Standards were framed compatibly. As a result of these analyses, broad categories of skills common to occupations within and across clusters were identified. This information was synthesized into task summaries to inform subsequent standards development activities.

Standards Development

During the second stage, representatives from health services, labor and education convened to draft standards for the health care core and for each occupational cluster. Separate committees were convened for each set of standards. Each committee was composed of individuals from across the nation, representing an array of occupational knowledge and expertise relevant to the health care core or the designated cluster. Drawing on the task summaries, the group discussion, and their own varied and substantial expertise, committee members formulated the initial version of the National Health Care Skill Standards, which was then subjected to review and validation.

Standards Review and Validation

To ensure conceptual soundness and broad applicability, the standards were extensively reviewed. This third stage was conducted in three ways. First, a 44-member Standards Review Committee convened to review the draft standards. The committee included a cross-section of representatives from professional associations and labor organizations, college educators, and practitioners from the various geographic regions and from varied health care delivery sites across the U.S. None of these participants had been involved in development of the standards, thus ensuring the independence of their review. Participants systematically reviewed the draft standards and made recommendations for improvement. These recommendations were collected, summarized, and analyzed, both quantitatively and qualitatively. The analyses were used to inform and focus the subsequent revision process.

Next, a mail survey was conducted, in which 40 targeted health care experts, educators, and practitioners across the nation were asked to evaluate the standards for their appropriateness and criticality. This survey allowed the NHCSSP to access individuals with particular expertise who had been unable to attend key committee meetings. The completed surveys were analyzed, summarized, and added to the project database.

Finally, over 25 focus group discussions with workers were conducted at 16 different health care delivery sites across the country to review the draft standards. Selected facilities varied according to a number of dimensions: large, medium, and small-sized facilities; proprietary and not-for-profit status; primary, acute, tertiary, and chronic care; urban, suburban, and rural facilities; small and large Medicaid or managed-care patient populations. Groups included both workers and supervisors. Most focus group participants felt the standards were sound, but that in some cases wording could be refined to more precisely convey meaning and context.

The combination of these different and independent review modes provided credibility and a richness of information to the review and validation effort. Project staff synthesized the information received throughout the review process to make revisions to the draft standards. As a final effort to gather input from as many individuals as possible, the previously published draft standards booklet included a short questionnaire regarding the appropriateness of the standards. Feedback from an additional 100 individuals responding to that questionnaire was added to the database and used during the final review of the standards presented herein.

At all stages of the development, review, and validation process, FWL invited the participation of practitioners, professional associations, and representatives of various sized and oriented facilities, from all parts of the country. This was to ensure that as many viewpoints as possible would be heard. Subsequent to each step of the review and validation process, and subsequent to the pilot implementation study, remarks and opinions were noted and integrated. The research methodolgy is summarized in Figure 5.

---

Figure 5 Research Methodology for National Health Care Skill Standards Project

---

In summary, development and validation of the standards was an iterative process, with each round of review bringing greater conceptual precision, as well as more terse and concise language. Additionally, commonalities across areas of work (such as those reflected in the Therapeutic/Diagnostic Core) became more evident through this process.

Pilot Implementation Study

During the second phase of the three-year NHCSSP, a small group of organizations agreed to pilot test the health care skill standards -- that is, actually use the standards for specific purposes in their organizations for a discrete period of time -- and report on both the implementation process and its results. The aim of this study was to gather information on the usability of the standards at the local level.

Twelve health care delivery and education sites participated in the study. Several sites included education-industry partnerships, such as a secondary school health program partnering with a hospital. Most sites piloted one or more standards-related applications, including curriculum design; inservice training; employee selection and hiring; and development of assessment tasks. While each site had its own particular goal and application of the standards, several common themes emerged. These applications and themes are discussed in detail in Part IV.