This latest volume of the Policy Support & Studies Program's Annotated Bibliography was prepared by Cindy McPherson and Jo Ann Izu. Its purpose is to provide readers with summaries of selected articles that raise some of the complex issues involved in designing accountability and incentive structures. It was funded in part by the U.S. Department of Education, Office of Educational Research and Improvement.


Contents:

Cornett, L.M., & Gaines, G. (1997). Accountability in the 1990's: Holding schools responsible for student achievement. Document #97501, $5.00, available from Southern Regional Education Board (SREB), http://www.sreb.org, (404) 875-9211.

Council of Chief State School Officers (CCSSO) (1998). State education accountability reports and indicator reports: Status of reports across the states 1998. Available at http://www.ccsso.org.

Danin, S.T. (1997). Implementing state mathematics standards in Delaware, New Jersey, and the District of Columbia. NASSP Bulletin, 81:590.

Education Commission of the States Information Clearinghouse (1997, December). Standards: When students get behind. Available at http://www.ecs.org.

Hill, P.T., & Lake, R.J. (1997). Toward a K-12 education accountability system in Washington State. Seattle: Washington University, Center on Reinventing Public Education. Available at http://www.gspa.washington.edu/CRPE/crpehome.html or ERIC document #ED418202.

Including students with disabilities in accountability systems. (1998, April). CISP Issue Brief, 3. Available from Consortium on Inclusive Schooling Practices, Alexandria, Virginia, (412) 359-1654 or ERIC document # ED418529.

Jones, K., & Whitford, B.L. Kentucky's conflicting reform principles: High-stakes school accountability and student performance assessment. Phi Delta Kappan, 79, 276-281.

Massell, D.. (1998, July). State strategies for building local capacity: Addressing the needs of standards-based reform. CPRE Policy Briefs, RB-25.

Massell, D., Kirst, M., & Hoppe, M. (1997, March). Persistence and Change: Standards-based systemic reform in nine states. Consortium for Policy Research in Education (CPRE) Policy Briefs. Philadelphia: University of Pennsylvania.

McCary, M. et al. (1997). Using accountability as a lever for changing the culture of schools: Examining district strategies. Available from SERVE at http://www.serve.org or ERIC document #ED408697.

Monson, R.J., & Monson, M.P. (1997). Professional development for implementing standards: Experimentation, dilemma management, and dialogue. NASSP Bulletin, 81:590.

Ruenzel, D. (1998, June). Shifting the focus to learning: California's accountability debates. EdSource Report. Available at http://www.edsource.org.

Ziebarth, T. State takeovers and reconstitutions policy brief. Education Commission of the States.

Cornett, L.M., & Gaines, G. (1997). Accountability in the 1990's: Holding schools responsible for student achievement. Document #97501, $5.00, available from Southern Regional Education Board (SREB), http://www.sreb.org, (404) 875-9211.

This report describes recent efforts in Southern Regional Education Board (SREB) states to increase local accountability for student learning, with particular emphasis on rewards and sanctions being used to strengthen compliance. In a series of detailed charts it shows how these fifteen states (AL, AK, FL, GA, KY, LA, MD, MS, NC, OK, SC, TN, TX, VA, WV) are measuring and reporting progress, identifies rewards being offered for student achievement, and describes early outcomes of sanctions and assistance programs.

The authors report that early studies suggest that many schools and school districts have found the transition to "90's style accountability," emphasizing local problem solving and decision making, difficult. Aligning curriculum with new standards demands a significant change and input of resources. State pressure to change can lead to practitioner resentment. To measure and report performance, states set an absolute goal and report student progress over time. Most states are using these reported gains to determine sanctions and rewards, either on a one year cycle, as in Texas and North Carolina, or a 2-3 year cycle (FL, KY, MD, TN). Some states have reported student achievement in categories ranging from "low performing" to "exemplary." Reporting to the public has been difficult, because of the drop in scores that comes with an adjustment to new standards, new tests, and new scoring systems.

To reward high scores, seven states offer cash to teachers. The numbers and amounts of awards vary, from a total of $500,000 in Tennessee, to $28 million in North Carolina. It is still questionable whether and how these external rewards for teachers impact instruction. For schools that do not present high scores, most states offer outside assistance and impose penalties for not improving. The process generally begins when the state defines a low-performing school, continues when experts help schools develop improvement plans, and culminates when the chief state school officer judges whether the school or district has met its goals. To accommodate this new approach, state education departments have reorganized toward a service rather than regulatory orientation.

To improve consistently low-performing schools or districts states may fire school staff, transfer students, close the school, consolidate districts, appoint an outside manager for the school district, or remove school board officials from office. To date, states have moved cautiously to implement such sanctions. Kentucky and Tennessee delayed action on low-performing schools until it could be determined that the measures used to classify the schools were fair and reliable. Maryland imposed sanctions and reconstitution on just a handful of the very worst schools. Most states that have taken districts over were responding to financial rather than academic issues. Early anecdotal evidence does suggest that sanctions create pressure on practitioners.

Three states--Florida, Kentucky, and Tennessee-have undertaken early evaluations of their new school accountability systems. A synopsis of the elements of each one is presented, followed by a list of five common issues to guide future efforts. The following recommendations are proposed. First, it is imperative to get tests and assessments "right" in order to get useful results. The consistent challenge has been that although new standards lead assessments in the direction of ones that are more open ended and performance based grading these "authentic assessments" on a statewide scale is daunting. On the other hand, states that continue to use old standardized tests do not measure progress toward new standards. In addition, the public needs to be able to understand and use the information gathered.

A second recommendation is to take action that is preceded by good planning and supported by evaluation. Pressure to get programs going has weakened implementation, with some states starting accountability programs before they had new assessments. In Florida, Kentucky, and Tennessee comprehensive and early outside evaluations provided helpful feedback to improve the accountability programs. Third, these programs must be easily explained to the public, parents, and educators. Tennessee encountered problems when nobody understood the way school scores were being calculated. Fourth, adequate help for schools and school systems must be provided. Most states have not provided enough resources for teachers, principals, and state department of education staff to build their capacity to implement desired changes. Rewards and sanctions may spark interest in change but cannot create it. Lastly, accountability programs must be comprehensive. They need to include standards, assessment, incentives, and school by school reporting to parents and the public. The public and educators need to understand how the whole system works, and ultimately whether it works to improve student learning.

< back to Contents >

Council of Chief State School Officers (CCSSO) (1998). State education accountability reports and indicator reports: Status of reports across the states 1998. Available at http://www.ccsso.org.

The Council of Chief State School Officers (CCSSO), a nationwide nonprofit organization of public officials who head education departments, worked with all fifty state departments of education to compile this information on the present status of state education accountability reports and state reports on education indicators. This report consists of a chart that identifies the name and types of reports generated in each state, along with dates, contact people and contact information. CCSSO has categorized the types of indicators and statistics contained in each state document in a separate report.

< back to Contents >

Danin, S.T. (1997). Implementing state mathematics standards in Delaware, New Jersey, and the District of Columbia. NASSP Bulletin, 81:590.

This article synthesizes three separate evaluations conducted by Research for Better Schools (RBS) to observe what was happening in a sample of classrooms in three states that had spent the previous three years developing math standards and frameworks. Delaware, District of Columbia, and New Jersey shared a process involving broad-based input from state and local stakeholders and guidance from national standards efforts. They used funding from the U.S. Department of Education and the National Science Foundation. They implemented professional development as a primary support to help teachers translate standards into practice in the classroom. RBS studied teachers' awareness and understanding of their state math standards and frameworks, the support teachers received to implement the new standards, barriers to implementation, and the perceived impact the changes had on teaching and learning. They make recommendations for supporting teachers to implement state standards and curriculum frameworks to improve student math achievement.

RBS selected participants based on their high level of staff development around the standards and frameworks. Not surprisingly, they found that all teacher participants were aware of the standards and frameworks in their respective states. Teachers did not, however, share a deep understanding of what to do with the standards, and they received conflicting messages, particularly due to the overlap of state and national (NCTM) standards. They were supported by a wealth of professional development opportunities and formal and informal teacher networks. Professional development focused on instructional methods and materials aligned with the standards. Teachers were able to choose conferences, courses, or workshops to meet their own needs, but they did not have a good overall picture of how the numerous pieces fit together. Teacher networks facilitated sharing and problem solving among colleagues.

There were problems with professional development, time, materials, and space. The impact of professional development for many was diffuse because it was not explicitly connected to standards and frameworks. In addition, to implement changes in their instructional practices, teachers needed more planning time and common planning time with colleagues. They also needed instructional materials and supplies, particularly for students with special needs. The availability of these materials was problematic. Lastly, teachers needed more storage space, different furniture, and alternate scheduling to implement fully the standards-based reforms.

Teachers did change instruction, reporting increased use of cooperative learning, problem solving, hands-on activities, calculators, and computer software programs in their math classrooms. They noticed positive attitudinal changes in their students in response to these instructional changes. Measuring the impact of reform on student learning, however, was more difficult. Teachers needed more tools to align assessment with the new standards. Alignment of assessment was also needed at the state and district level to provide clear feedback on the influence of standards-based reform on student achievement. Teachers in Delaware and D.C. worried that their students might not do well on state tests, because the traditional standardized assessment doesn't measure new practices in math.

The authors offer three recommendations for standards-based math reform. First, assessment systems must be established in alignment with standards. Second, professional development must continue and be explicitly linked to state standards, frameworks, and best practices. Third, states and districts must provide teachers with the necessary classroom materials to do their job.

< back to Contents >

Education Commission of the States Information Clearinghouse (1997, December). Standards: When students get behind. Available at http://www.ecs.org.

This policy overview describes a problem with how states support students who do not meet minimum education standards, and it discusses six policy solutions that have historically been implemented to help students who achieve below set standards. These solutions include alternative curricula, extended day programs, remediation and retention, sanctions/incentives, and ungraded classrooms. Relevant legislative and policy history from more than ten states is included. Issues in assessment for students with special needs are also presented. The author's position is that policies should acknowledge individual differences and provide for those students who cannot pass the test in a way that is productive for their education.

According to this article, the problem is that services to students who fall below minimum standards are inadequate and inequitable. The American Federation of Teachers (AFT) reported in 1996 that only ten states required and funded programs to help low-performing students reach state standards. Eight more states required and did not fund this intervention. Urban and minority students are more likely to fall behind than white or rural students, at least in part due to the inferiority of schools that serve those populations. Complicating the issue is inconsistent or under-reporting of state dropout rates. Further, the number of students labeled learning disabled has risen sharply over the past twenty years. Given that nearly half of those students may be excluded from state testing programs, the learning disabled and special education tracks potentially become dumping grounds of low-performing students in districts where pressure to elevate test scores is high.

States that have policies requiring and funding intervention to bring students up to grade level are more likely to be successful at making standards-based reform count for all students. Following is a description of several program options in practice. Alternative curricula, implemented at the school or district level, blend academic coursework with vocational training. Differentiated diplomas are offered in eight states to recognize students achieving at different levels. Academically oriented extended day programs offer students additional instruction and assistance. Remediation sometimes improves achievement when it happens in early grades and is targeted to serve the individual. Retention has been eliminated in many states, and now social promotion is disallowed by some. Precedent from twelve states is summarized, as are two special school-based intervention programs, AVID (Advancement Via Individual Determination) and Success for All. A number of states also have legislation involving incentives and/or sanctions, both for students and teachers, as a means to raise academic performance. Recent studies question the impact of external rewards for teachers. Ungraded classrooms have demonstrated success in early grades in KY and WA, however, detractors say they place an unfair burden on teachers.

At present, students classified as emotionally or learning disabled are excluded from assessments in forty-one states. Parents, students and teachers may request their exclusion, and schools can waive them out. New Title I legislation, however, mandates that special needs students be included in state and district-wide assessment by 2001, with accommodations as necessary. Requirements for instructing other students with special needs, such as those with limited English proficiency (LEP) vary considerably among states, with those who receive insufficient services suffering academically. Clearly, states need to address these issues.

< back to Contents >

Hill, P.T., & Lake, R.J. (1997). Toward a K-12 education accountability system in Washington State. Seattle: Washington University, Center on Reinventing Public Education. Available at http://www.gspa.washington.edu/CRPE/crpehome.html or ERIC document #ED418202.

This article proposes a model of accountability for educational reform in Washington State. It defines accountability as a process to improve schools for all students, which must accomplish three tasks: attach consequences to performance (provide strong incentives to improve instruction); create flexibility to empower principals and teachers to make the changes they need in order to improve; and ensure that schools choose and get the assistance they need to reach the goal. Hill's theory is that school level initiative must be the engine of reform. He lists and discusses seven lessons learned from other state accountability efforts and makes seven more recommendations for a system in which the whole community-- business, private sector, public education-makes an effort to improve Washington's schools. The article includes several useful charts: a diagram of how the system would work; a table outlining the responsibilities of each player in the game, from the governor to the school; and an appendix that compares the designs of state accountability systems in TX, KY, MD, SC, and TN.

Experience suggests a number of elements for success in accountability systems. First, responsibility and action should be located at the school level. Teachers and principals need to make decisions that matter, be able to reallocate funds, and believe in the changes being implemented. Second, schools that are struggling need to be able to get assistance that is relevant, responsive, and available when they need it. Third, school boards deserve a strong role in the process. Experience in Kentucky, Texas, and Tennessee has demonstrated the weakness of centralized state systems that ignore the role of local communities. Fourth, direct measures of student learning must be reported. The performance of all students--regular, special ed., and bilingual--should be considered, and student performance should not be buried in a whole school score. Fifth, school classification should be driven by numbers but should leave room for judgment based on the school's needs, past performance, and ability to improve. Sixth, there needs to be an independent oversight body, representing all interest groups and exerting authority to implement the accountability system. Finally, it is appropriate to reward whole schools for their success and give struggling schools help, but continued low-performance should not be tolerated. Hill recommends targeting state resources strategically so that assistance is short term and powerful.

Hill proposes a statewide accountability system with the following elements. It gives schools the flexibility in decision making and the financial support to make tangible school level instructional improvement. It stimulates the development of a range of public and private sources of technical assistance to schools. It measures and reports real, unadjusted student achievement scores. It puts school districts in control of classifying, rewarding, assisting, and/or reconstituting schools. It realigns the responsibilities of existing state agencies to serve the system. It creates an independent state accountability commission with the authority to monitor school districts' progress and recommend reconstitution for schools or districts that continue to fail. Lastly, it allows teachers and principals to hold students accountable for trying and behaving. Students who do not adhere to these two minimum standards would be moved to newly created alternative schools. In sum, the proposed accountability system holds students and schools to high standards and clearly delineates the responsibilities of each player in the system.

< back to Contents >

Including students with disabilities in accountability systems. (1998, April). CISP Issue Brief, 3. Available from Consortium on Inclusive Schooling Practices, Alexandria, Virginia, (412) 359-1654 or ERIC document # ED418529.

This policy brief defines accountability and presents six different implementation approaches identified by Michael Kirst. It presents a framework to analyze state and local accountability policies with respect to students with disabilities. Accountability is broadly defined in this article as the use of systematic methods to inform those inside and outside the educational system that schools are moving in desired directions. What need to be more narrowly defined are the roles and responsibilities of those involved in school improvement. Accountability systems to date have focused on performance reporting, monitoring and compliance, incentive systems, reliance on the market (school choice), changing professional roles, and/or changing the locus of control of schools. Students with disabilities have not usually been included in state assessment and accountability systems. Generally, though, accountability within special education has focused on monitoring and compliance. A significant political change is that in 1997 the Individuals with Disabilities Education Act (IDEA) was reauthorized to require that students with disabilities be included in state or district assessments or be given an alternative examination. This brief looks at perspectives on this issue, assessment of students with disabilities, consequences, and issues still to be resolved.

Foremost, including students with disabilities in state assessment increases the integrity of the system because it counts all students. The new amendments to IDEA require that schools set performance goals for students with disabilities, that they include these children in state assessments, with accomodations, and that they report progress to parents as they do for nondisabled students. The amendments do not require that students with disabilities be included in state accountability systems. In other words, unlike race/ethnicity, gender, or economic status, special education will not be a subgroup for which districts and schools are held accountable. Descriptions of state approaches to include students with disabilities in accountability systems in Delaware, North Carolina, and New York are presented. Local approaches in San Diego and Rochester are also described. The author points out several issues that still need to be addressed by states and localities.

One of the most significant issues for states to consider is establishing shared content standards for students with disabilities. In order to include these students in accountability systems, states need to ensure that they focus on teaching and learning for all students, compile data on student learning from student performance, collect data on the number of students excluded from state assessment, and maintain flexibility in considering outcomes for students with disabilities. The accountability system must include standards for students with special needs, and monitoring and incentives for compliance. School districts concerned with including students with special needs in local accountability need to maintain results-oriented data collection focusing on services to students, publicly report information on students with disabilities, and develop data systems to track the performance of students in inclusive and segregated, or pull-out programs. Members of the community need to be included in reviewing student performance, and accomodations made for students with special needs in testing should be documented for others to use.

In conclusion, this piece calls for a well-designed accountability system that combines school context, processes, and student performance to guide school improvement efforts. The system must be grounded in authentic data about students and resources, and it must include all students fairly and equitably.

< back to Contents >

Jones, K., & Whitford, B.L. Kentucky's conflicting reform principles: High-stakes school accountability and student performance assessment. Phi Delta Kappan, 79, 276-281.

The assertion in this article is that Kentucky's high stakes accountability system has undermined the instructional benefits of student performance assessment in state educational reform. The authors reflect on the idea of the Kentucky Instructional Results Information System (KIRIS), detail its evolution, and point out its effects. They question the use of performance assessment for high stakes accountability and advocate an alternate strategy to inspire professional commitment and accountability, rather than forcing compliance through bureaucratic control.

The Kentucky Educational Reform Act (KERA) began with the belief that all students can learn at high levels. It called for teaching and learning that is authentic, emphasizing instruction that requires problem solving, reasoning and communicating in real-life situations. It mandated the development of a new performance-based testing system, aligned with the new standards for teaching and learning. Finally it connected a high-stakes accountability system to student performance, turning student test results into a school score used by the state to determine rewards and sanctions for teachers and administrators, in order to compel them to implement the reforms.

The following summary tracks the revision of KIRIS over the first six years of the program. Originally, the state defined six learning goals and elaborated 75 "valued outcomes," which were scrutinized by various interest groups. They were revised, removing goals relating to social competence, and renamed "academic expectations." It became clear that these objectives did not provide enough detail on both content and performance standards, so the state responded with a draft document, Core Content for KIRIS Assessment. Meanwhile, state committees worked to develop the new performance-based assessments. Since these assessments were linked to high stakes accountability, reliability of scoring of the instrument became an overriding concern. In the interest of reliability, the emphasis shifted from performance events and portfolios, to open response questions, and then from open response questions, which require more judgment in scoring, to multiple choice items. At the time of publication, multiple choice test questions had been reintroduced, while performance events and the math portfolio were removed from the test.

KERA attempted to empower schools to make more local decisions about curriculum. It mandated the creation of school-based councils, but did not require implementation of this system until 1996. Schools have been constrained by the requirement to develop School Transformation Plans, which are required for schools who do not meet their target scores or who are applying for state grant funding. A rebound effect occurred as schools, pressured to do well on the test, pressed the state to give more specific content guidelines, which constrains local decision-making about curriculum. Schools are further constrained by the lack of individual validity in the state assessment, which makes it impossible to measure student accountability. Instead, schools receive scores based on student performance on the test. Further, different groups of students are tested each year, so progress is not measured in one group of students, but between groups, from last year's fourth graders to this year's, for example. Many educators lack faith in this system, which is complicated by an arbitrary formula that sets an expectation that schools will demonstrate a constant rate of improvement (one tenth of the difference between their baseline and their target), every 2-year cycle for twenty years. Teachers are rewarded for higher student scores with cash bonuses, and schools that come in under their goal receive assistance or takeover from a "distinguished educator." The overall climate is that schools are intent on raising test scores.

Though these system changes are documented, it is too soon to tell whether KIRIS has improved student learning. The effect of KIRIS on teacher-student relationships and instructional practices is notable. The authors assert that the state test has inhibited productive teacher-student relationship and curtailed teacher professional judgment by driving the creation of a state curriculum and holding teachers accountable for student performance. The emphasis on testing learning that can be measured reliably and validly has driven the curriculum so that performance assessment is not incorporated into classroom practice. In addition to criticizing the link of high stakes accountability to assessment, the authors question the premise of using performance assessment for high stakes accountability at all. Inherently, performance assessments are meant to demonstrate learning in an individual and not to be aggregated.

The authors conclude that the Kentucky system of high stakes accountability has raised important questions about teaching and learning, but has not and can not answer those questions. They envision an approach to accountability that encompasses professional practices, equity issues and opportunities to learn, as well as student test scores. It would be a collaborative effort inspiring teacher commitment and professionalism.

< back to Contents >

Massell, D. (1998, July). State strategies for building local capacity: Addressing the needs of standards-based reform. CPRE Policy Briefs, RB-25.

This policy brief examines strategies used to build capacity, or what it takes to translate high standards and incentives into effective instruction and sound student performance, in eight states (CA, CO, FL, KY, MD, MI, MN, TX). Information was gleaned from many interviews with policymakers in each state and examination of supplementary background materials. The researcher groups strategies as either classroom level or school, district and state level. She identifies seven types of strategies that have been used in various amounts and combinations. In the classroom, capacity-building strategies address teacher knowledge and skills, student motivation and readiness to learn, or curriculum materials. At the organizational level, factors are taken into account such as the quantity and types of people supporting the classroom, quantity and quality of interaction within and among organizational levels, availability of material resources, and organization and allocation of school and district resources. Four capacity-building strategies all the states had in common are presented and discussed, followed by a critique of what has worked and what challenges remain. The author concludes with a checklist for policymakers to consider when designing their capacity-building systems.

The most common way states provided professional development and technical assistance was by building the external infrastructure to provide these services. Several states in this study used regional service centers like county offices of education to serve specific programs, districts, and reform goals. They also relied on professional networks of teachers and other experts to improve the knowledge and skills of some and train or assist others. A few networks developed and distributed products like lessons, assessments, and scoring rubrics to assist teachers. California's well-known Subject-Matter Projects effectively increased teacher reflection about their practice and provided time and assistance to develop instructional strategies and projects. Professional associations and institutions of higher education also stepped in to provide professional development, curriculum support, and assistance in developing state policies.

A second strategy that all states used was setting standards for professional development and preservice teacher training. Texas took steps to hold whole institutions accountable for the performance of teacher education students. Providing curriculum materials was the third common practice. Most states developed broad standards documents, leaving room for local practitioners to fill in the detail. Frequently, however, school districts complained that they lacked resources, time, and expertise to translate broad state standards into practice. States responded to the concern by writing more detailed curriculum frameworks, setting up resource banks to help disseminate instructional materials, and supporting or showcasing effective programs. Fourth, states decentralized resources, linking them either to school site decision making or to accountability systems. Through both avenues, schools were required to develop improvement plans.

Strategies that show promise are decentralizing assistance to schools, nurturing professional networks, providing curriculum guidance, and setting professional development standards. Numerous potential challenges have also emerged. External service providers may not have the fiscal or human resources to serve all teachers and schools. Student performance data need to be incorporated into accountability systems in ways that are fair and understandable. Middle performing schools need support. Continuity and stability need to be maintained, even during periods of public debate over new strategies. Incentives to heed professional development standards, pursue professional development, improve teacher training, maintain high expectations for all students, and engage in the school improvement process need to be put in place.

< back to Contents >

Massell, D., Kirst, M., & Hoppe, M. (1997, March). Persistence and Change: Standards-based systemic reform in nine states. Consortium for Policy Research in Education (CPRE) Policy Briefs. Philadelphia: University of Pennsylvania.

This article gives an overview of the evolution of standards-based, systemic reforms in nine states (CA, CT, FL, GA, KY, MN, NJ, SC, TX) between 1994-1995. It discusses the following emergent themes: steady, incremental progress, balancing new and old approaches, building capacity for reform, and addressing equity issues. The research is based on interviews with policy makers and educators in the nine states, at state and district levels.

The first theme the authors discuss is the slow and steady rate of change. All states studied developed or revised their academic content standards, although some states took as long as five years to agree on the form and content of these documents. Many states focused on improving statewide assessment programs and worked to align assessment with standards, integrate performance-based assessment, and reduce norm-referenced testing. Progress was also made toward building capacity of teachers, through revision of teacher credentialing, attempts to align professional development of experienced teachers with state standards, and formation of subject-matter or grade level networks, supported by the states, universities and other agencies. Support for this progress came from the education community, business communities, and national associations like the National Science Foundation. Opposition to standards-based reform initiatives came from traditional Christian and conservative groups. "A critical element facilitating the continuation of standards-based reform," the authors state, "was the effort by policymakers to establish a balance between often-competing poles of reform" (p.6).

What needed to be balanced was the new with the old, or the innovations with the emphasis on basic skills, familiar teaching methods, and conventional testing. To make the reforms palatable, policymakers modified, but did not completely toss out either the new practices or the old ones. Districts also pursued a balanced approach, combining performance assessment with traditional assessment, and cooperative learning with independent learning. In general, local school districts saw state standards as one resource in their process of improving curriculum. Many districts elaborated on state standards and were not uniformly driven by state action. Educators typically wanted detailed guidance and support beyond what the state documents offered.

The new standards put teachers into new roles, requiring new skills and knowledge. The need to help practitioners build capacity for implementing reforms began to be addressed a few years into the standards-based reform movement, after standards and assessment documents were developed. Still, states have not been highly directive about or supportive of professional development activities, and professional development monies have frequently been cut in times of budget reduction. Another issue that has not received enough attention in the first stages of this reform movement is equity. Though the rhetoric in support of standards is about increased challenge for all students, the policy efforts for reform have been focused on the general education population.

In light of their findings the authors raise several concerns. First is whether the approach of balancing old and new methods will actually facilitate thoughtful changes in schools or send mixed messages about reform. Second is the persistent need for capacity building. Teachers and administrators need more support for carrying out standards-based reform. Third, the long-term viability of these reforms depends on addressing standards for and being accountable to students with disabilities. Resources must be channeled to address how to serve all students in challenging academic environments. Further, societal problems such as poverty, hunger, homelessness, violence, and addiction must also be addressed, because they significantly impede academic progress.

< back to Contents >

McCary, M. et al. (1997). Using accountability as a lever for changing the culture of schools: Examining district strategies. Available from SERVE at http://www.serve.org or ERIC document #ED408697.

In this document from the Southeastern Regional Vision for Education, a school district superintendent presents a description of locally owned accountability and suggests strategies for understanding and implementing accountability for quality. In the first chapter he lists several unintended consequences of high-stakes accountability. When testing has been used to grade a school, teachers in low-performing schools have become demoralized, the curriculum has been narrowed and teacher professionalism has been reduced in order to focus on raising scores on the test. Students have suffered from unethical placement practices that artificially raise test scores (grade retention, placement in special education) and from reduced participation in higher level courses (ones that do not serve the goal of raising test scores). McCary prefers an alternative, grassroots internal school accountability model. He provides an example in the tale of Elizabeth City-Pasquotank, a small rural school district in eastern North Carolina. The strategies that were successful in this area are summarized and categorized according to whether they built whole school, teacher, or student commitment to quality. The appendices include samples of district wide school improvement planning process indicators, staff feedback forms, and components of an elementary grades skill building program in communication.

From the North Carolina experience, a number of facilitating factors emerged. For instance, the central office leadership was committed and provided clear goals to guide the accountability process. The school board supported change. Resources and decision making were located at the school level. Personal relationships between schools and their central office developed since the district is small. Improvement plans were informed by research. Professional development in instruction and assessment was ongoing. A commitment was made, from the top down, to honest reflection, good listening and learning from mistakes. This story provides a vivid example of an internal accountability process. Pieces of the model may apply in other contexts; however, this is not an evaluative study.

< back to Contents >

Monson, R.J., & Monson, M.P. (1997). Professional development for implementing standards: Experimentation, dilemma management, and dialogue. NASSP Bulletin, 81:590.

In this article a school district superintendent and an educational consultant reflect on three ways professional development can be targeted to support practitioners implementing standards-based reform, particularly in understanding performance assessments at the high school level. They discuss three approaches and provide numerous examples of professional development activities that respond to dilemmas encountered in using performance assessments.

The authors acknowledge that working with standards and performance assessments can be challenging, and that professional development is needed to build capacity in schools to successfully implement new curriculum standards. Their first belief is that professional development experiences need to be organized for teachers to experiment with designing and implementing performance assessment in their classrooms, as well as interpreting the information these assessments generate about student achievement. As questions come up, the focus of professional development can turn to thinking and acting strategically to solve problems. Finally, teachers must have a forum to reflect on the dilemmas encountered during their trials with performance assessment. Administrators should promote the opportunity for dialogue within the individual and among faculty, students, parents, and university faculty. The call to administrators is to use these recommendations to structure successful transitions to standards-based curriculum and performance assessment.

< back to Contents >

Ruenzel, D. (1998, June). Shifting the focus to learning: California's accountability debates. EdSource Report. Available at http://www.edsource.org.

This EdSource report focuses on what several prominent education experts and school superintendents identified as central challenges and opportunities in the current movement for an accountability system in California. It summarizes a discussion that took place at the April 1998 EdSource conference with the same title.

Gerald Hayward, Co-Director of Policy Analysis for California Education, cautioned the audience that although accountability is a state priority in education reform, it is complex and will take time and money to implement-perhaps up to ten years and hundreds of millions of dollars. The key issues he sees in any proposed accountability system are that it set clear and rigorous standards, align curriculum, assessment and teacher preparation with those standards, establish positive and negative consequences for schools, and stay flexible enough to be adjusted in progress. The recommendation of the Rewards and Interventions Committee in Steering by Results, which Hayward summarized, are that schools be evaluated based on the results of a new state assessment, high performing schools receive cash rewards, low-performing schools receive assistance, and failing schools be taken over or closed. The long-term goal is that 90% of students reach or exceed grade level standards. In the short term, schools will be held accountable for showing growth toward that goal.

Michael Kirst, professor at Stanford University and Co-Director of Policy Analysis for California Education, reminded the audience that Californians have already seen a number of attempts at accountability: "Cash for CAPs" in the 80's and CLAS in the 90's. The biggest problem he sees in failed systems is their top down approach. It is crucial for any state accountability system to inspire teacher commitment. It must, therefore, build internal accountability, addressing the intrinsic motivation teachers have to see their students succeed. This type of system, based on new standards, requires that teachers acquire a much deeper knowledge of their content and methods. Kirst is concerned that the system proposed in Steering by Results does not address these real issues.

Elaine Johnson, past president of the California Council on the Education of Teachers, spoke on behalf of teachers. Teachers' hesitation toward accountability is that it appears to focus solely on quantitative aspects of education and unfairly hold teachers responsible for public education's failings. She asserts that accountability is contextual, and teachers are not always in control of student learning. Further, emotional and psychological awareness needs to be considered. The problem with the state's current testing program is that it does not measure what teachers are teaching. To hold teachers more accountable, she emphasized teachers need more and better professional development and support.

Scott Hill, Executive Director of the Commission for the Establishment of Academic Content and Performance Standards presented the audience with a preview of some of the new standards being written. He noted, like Hayward, that the process of fully implementing standards aligned with curriculum and assessment will take five to ten years. Standards alone will not change the system. The issue of how students can be held accountable for the standards still remains to be determined. Hill feels that they should be responsible not only for showing progress, but also for meeting minimum competency. Whether it is fair to base accountability on a single test, or how much influence teachers should have in assessing student performance are just some of the questions likely to engender debate in California.

To balance the discussion, three superintendents spoke about the district role in implementing accountability systems. Ron Hockwalt's Walnut Valley school district, east of L.A., put academic standards in place. They spent two years developing teacher capacity in science. They also improved teacher evaluation. Marcia Plumleigh's Campell Union Elementary School District, in Silicon Valley, took ten years to develop a district-wide performance-based assessment system. This too required extensive capacity building through teacher professional development. Long Beach, led by Carl Cohn, developed standards, assessment, and a professional development center independent of the state. They also developed interventions to assist low-performing students, requiring summer school, an intensive literacy program, and even a pre-high school remediation academy. Cohn's assertion is that the state can not really guide accountability in a way that makes sense. Educators do not regard incentives like financial rewards or school takeovers effective. For this reason, local control should drive accountability. All three district leaders pointed to the need for adequate resources to address growing demands for accountability. Despite the odds, though, they also asserted that schools are currently doing an extraordinary job.

From this conference, it two points are clear: attempts to develop and implement meaningful accountability systems should move forward, and policymakers, educators, and the public need to know that the process will take time, money, and perseverance. Six questions will need to be answered. Can educators and policymakers agree on a fair set of performance factors by which to evaluate school performance? Do practitioners have the capacity to implement the new standards? What state policies on professional development will be most effective at building capacity? What additional resources will school need? Will financial rewards and interventions be effective? What balance of incentives and consequences will most successfully hold students accountable for their academic achievement?

< back to Contents >

Ziebarth, T. State takeovers and reconstitutions policy brief. Education Commission of the States.

This policy brief describes and discusses two recent and controversial accountability approaches: state takeovers and reconstitution. It examines early effects of these procedures and raises questions for policymakers. It recommends that policymakers look closely at the criteria used to identify and enforce these options, the decisions they make about best interventions, how they implement takeover or reconstitution, and how they could effect more long term changes in low performing schools. Tables summarizing state laws and actions are included.

State takeovers are designed to occur when the state legislature, state board of education, or a federal court charges the state department of education or another agency with managing a local school district for a period of time. Takeovers occur only after repeated warnings and interventions have failed to improve a school district. So far, twenty-two states have passed legislation allowing takeover of school districts for academic reasons, and at least ten states have actually taken control of a low-performing school district. The balance of state and local influence in takeovers varies among states. There are two opposing sides on this issue.

Those who support state takeovers see them as a necessary extension of the state responsibility to provide public education. Furthermore, takeovers supposedly allow state and local leaders to combine resources, allow a competent executive staff to guide improvement efforts, catalyze public involvement, allow radical and necessary change in school districts, show school boards they need to improve, and use achievement data collected from school districts to strengthen accountability efforts. Opponents, however, purport that takeovers increase state authority at the expense of local control, falsely assuming that states can effectively run school districts, and putting poorly prepared elected officials in charge of schooling. They complain that districts are targeted for takeover based on narrow definitions of student learning gleaned from standardized test scores, and that such actions do not go to the root of the social problems disadvantaged students in urban districts face. Takeovers decrease morale among students, parents and practitioners, and they pit local and state officials against each other.

There is little research to verify these opinions on the effect of state takeovers. In a few cases management practices have improved, however, academic achievement is mixed at best. In addition, implementation of takeovers has been challenged by the U.S. Department of Justice on grounds that it violates voter rights to elect local officials.

Reconstitutions are different from takeovers in that control remains local, and districts are empowered to replace principals, teachers and other school staff, create new philosophies and new curricula. Schools in eight states have been reconstituted, beginning with schools in San Francisco in 1983. Reconstitutions have been implemented to address chronic problems such as low attendance and high drop out rates, low performance on standardized tests, failure to improve student performance, poor morale among school community members and deteriorating school buildings. Advocates of this approach believe it indicts school organization and culture rather than faulting individuals, it stops bad education, and brings in eager staff. They believe that the learning environment in a school can be changed by changing personnel, fostering a new culture where school failure is not tolerated. Through reconstitution, schools that face overwhelming problems should be able to redirect instructional practices and boost accountability efforts.

Those who oppose reconstitution contend that it has been implemented inconsistently, focusing on bad people instead of bad practices. They fault it for demoralizing school community, placing new staff in a hard situation, discriminating against poor and minority children by failing to take into account larger problems in the community, and undermining reform efforts already in progress. Opponents posit that ineffective instruction needs to be changed as well as ineffective school staff.

Again, unfortunately, evidence about the effects of reconstitution is scant. In some cases it has brought order, stability and public involvement to low performing schools, but again, academic progress is uneven. The threat of reconstitution may be a motivating force for improvement, but the effectiveness of reconstitution has not been proven. After raising questions for policymakers to consider on both these issues, the author concludes that the effects of state takeovers and reconstitution are debatable and that more effective approaches for assisting low-performing schools and districts are needed.

< back to Contents >