back to list of Papers
back to list of Papers
Bill Clune, February 5, 1997
The research of the Policy Team of the National Institute for Science Education (NISE) has concentrated in three areas: implementation of systemic reform, implementation of the national standards, and equity. I would like to report findings and key research questions in each area and then turn to what we are learning about the links between systemic reform and student outcomes and the implications for policy and research.
(1) The Knapp synthesis of research on implementation of systemic reform
On the first topic of implementation, we commissioned a paper by Michael Knapp which he entitled, "Between Systemic Reforms and the Mathematics and Science Classroom." This paper synthesized research not only on NSF's systemic initiatives but other systemic reforms such as in Michigan and California. The paper found that systemic reform had stimulated widespread planning and discussion, in other words, had become a major presence on the policy landscape; but had produced only incremental change in the classroom, that is, an absence of broad and deep or systemic change. The key research question is which strategies lead to wholesale as opposed to selective curriculum replacement. By "curriculum replacement," of course I mean the implemented curriculum, as actually taught in schools
(2) Findings on the national standards and local implementation
The findings on local implementation, or implementability, of the national standards were complementary. The standards are very general and comprehensive, which allows localities and textbook writers to claim compliance without making much real change. There is a tension between constructivists who emphasize problem solving and traditionalists who prefer formal analysis. There is real confusion about how to define deep understanding of the subject matters and so-called habits of mind, as well as political controversy over content and consequences, such as for the traditional group of college bound students. A key knowledge gap is the availability of packages of curricula, assessments, teacher development materials, and technology (such as interactive learning) which fit the standards, as well as alternative educational goals and philosophies, including both college prep and vocational goals. In other words, it looks like some of the disputes cannot be resolved by consensus but instead will require the design of highly developed curricular options. Relevant knowledge of this kind exists within NSF and many sites but has not been pulled together adequately and connected with systemic reform.
(3) The case of the Virginia standards and the importance of non-politicized review
It is important that we do serious quality reviews of the entire chain of developing and implementing standards, as illustrated by the politics now surrounding the Virginia standards. If these standards are indeed of high quality and are appropriate for certain educational goals and philosophies, that conclusion should be firmly established and widely publicized by independent researchers. On the other hand, if alternative models are needed, those, too, should receive appropriate suppport. And there is an important issue beyond disputes over competing goals and philosophies, because politicization can present dangers for any kind of standards-based reform. If a particular set of standards is not appropriate for all educational goals and philosophies, a political "victory" for one group may be costly to many students; and symbolic politics over standards may actually prevent us from even looking any real development of the implemented curriculum and teacher capacity.
We should not have expected that the search for "universal high standards," or "hard stuff for all kids," would be easy; but neither should we leave this central question to politics. Perhaps international comparisons, such as TIMSS, can shed some light on options for implementing universally high standards.
(4) Findings on equity
We had two papers on educational outcomes in math and science by race and gender by Bill Tate and Alberto Rodriguez. These updated and refined similar work done in the past. They found many substantial differences across sub-groups of race, ethnicity and gender; and mixed evidence for the proposition that similar course taking will produce similar results. Such courses narrow the achievement gap but don't eliminate it and are not available to many students. Clearly, the effort to measure and report on gaps in equity yields valuable information.
But we also encountered serious debates about equity indicators. One is whether less academic courses are needed for more equity, or whether we should stick with college preparatory courses. This debate appears to be occurring in the field through a conflict of contending forces, rather than through rational design and options. A related debate is whether the student assessments used to measure achievement gaps are the correct measure of equity and especially the correct criteria for selective admissions. Thus, the key research question is how to simultaneously raise standards and broaden educational access.
(5) The need for theory linking process and outcomes (what we need to know from
research on systemic reform)
Across all three areas, we need a better theory linking the process of systemic reform, or policies, with outcomes. Theories which link process and outcomes in education, or any area of social science, are valuable and rare. It took a long time, example, to establish the connection between course taking and achievement to the point that we now consider course taking an intermediate outcome, as much as a process variable.
(6) Key process components of successful systemic reform: A change strategy
controlling:
If we look across sites attempting systemic reform, the more successful sites have a change strategy involving control over (and through) the power structure, curriculum and teacher development, school improvement, public acceptance, and data on incremental changes in curriculum and achievement.
(7) Critical flaws in real change strategies
On the other hand, unsuccessful sites have one or more of a familiar list of flaws: lack of political integration and will, political vision without a pedagogical delivery structure, school improvement or teacher enhancement projects with no realistic strategy for scaling up, external accountability or technical assistance without internal buy-in, unmanageable public controversy, perhaps over a previously low-profile professional agenda, lack of resources and the limits of volunteerism, and the absence of a planned strategy for incremental change in schools and classrooms.
(8) Implications for research and systemic initiatives
If the conditions for successful systemic reform are really much more stringent than previously suspected, what are the implications for any policy seeking to expand the scope of such initiatives? First, the profile of success would need to be clarified, so that sites could decide if the necessary elements were in place. Realistically, there probably would be several profiles, such as policy plus delivery and networks plus scaling up.
These profiles would have to be translated into initial conditions for participation; but if we want anyone to accept the conditions, the incentives for participation should match the intensity of the commitment. If we expect that a full-scale systemic reform will be built from scratch, serious attention should be given to the resources which could support this kind of effort, instead of expecting a "moonshot on a shoestring."
Perhaps the most interesting bottom-line policy question is the cost-effectiveness of partially successful systemic reforms. Suppose that nothing like full-scale systemic change will be produced, in the sense of wholesale replacement of the curriculum and substantial gains in achievement for all children. But also suppose significant progress in both process and outcomes, namely, that a system has been built which is capable of producing continuous improvement in curriculum and instruction, with assurances of socially valuable change in curriculum and achievement. Then suppose that the initial investment is not very large, as is usually the case in systemic initiatives.
That could present the situation of failure of systemic reform in the large sense but success of a systemic-type reform as judged by cost-effectiveness. The key research question would be when there are enough pieces of a complete reform to produce some significant results at a relatively modest cost, in other words, when a reform crosses some as yet undetermined threshold of cost-effectiveness.
Nothing is more common in policy than ambitious rebuilding projects being scaled down into modest home improvements (except perhaps for complete failure), and we should not lose sight of the broader objectives, because they do appear feasible in some places. On the other hand, I know of nothing in policy which should prevent us from accepting cost- effective policies, especially in this vital area of social policy. At base, it is the old question of the half-full or half-empty glass, with the key question being just how half-full and what is the cost of the drink. Answering that question will require a kind of research which is appropriately skeptical of optimistic claims yet sensitive to the importance of small, but important, variations in actual performance.