SUMMARY

        Accountability refers to the practice of holding educational systems responsible for the quality of their products. Most formal policy initiatives in this area focus on highly aggregated accountability at the state or federal level, but it is also possible to define accountability systems in local terms. The Carl D. Perkins Vocational and Applied Technology Education Act of 1990 attempts to promote both state and local accountability, placing with states the final responsibility for establishing measures and standards and for ultimate program supervision after placing initial responsibility at the local level for program evaluation and improvement.

        To understand the impact of this model of accountability, it is important to know how vocational programs presently function with respect to their multiple constituencies. The purpose of this study is to determine the extent to which local accountability systems exist in vocational education and to describe the nature of the underlying relationships between such programs and their constituents. To explore local accountability, we visited secondary and postsecondary vocational programs in five states: California, Florida, Michigan, Ohio, and Oklahoma. We gathered documentary information about programs, and we interviewed students, parents, instructors, employers, and administrators. In addition, we interviewed staff at the state departments of education. The interviews focused on the relationships between the programs and their local constituents.

        Four major conclusions emerged from our investigations. First, there is widespread evidence of functioning local accountability systems in vocational programs. Second, these accountability systems can be described in terms of four elements--goals, measures, information feedback loops, and change mechanisms--and the relationships between them. A simple model based on these elements does a reasonable job of describing accountability relationships across a wide range of vocational programs. Third, the quality of these components and the relationships between them account for much of the variation in local accountability systems. Limitations in these components interfere with the overall effectiveness of the accountability system. Fourth, we can identify many practical constraints that reduce the effectiveness of the components in a local accountability system. This may provide a basis for developing criteria to evaluate local accountability systems and prescriptions for improving them.

        These constraints and limitations can be described as deficiencies in the components of the model and the interactions among them. Specifically, goals were ineffective when

• Higher-level goals were not supported by interlocking goals at the action (local) level;
  
  
• Goals were sufficiently broad or vague that it was impossible to know when or if they had been achieved;
  
  
• Relevant constituencies did not understand and/or support the goals; and
  
  
• There were unstated or unclear priorities among goals.

        The chief role that measures play in accountability is to provide evidence of the attainment of goals, and the most important measures are those that are goal-related. Measures were ineffective when they were

• Not consonant with or sufficient to encompass goals;
  
  
• Of inadequate technical quality; or
  
  
• Not meaningful to constituents.

        Feedback includes the flow of information conveyed by the measures to administrators, program staff, and school system constituents, as well as the flow of information among administrators and staff. Potential deficiencies in feedback include

• Insufficient communication;
  
  
• Inaccurate communication; and
  
  
• Low signal-to-noise ratio--a high rate of communication with very little useful content.

        Ultimately, accountability systems lead to organizational change. Organizational reform mechanisms can falter for many reasons, including

• Regulations that limit options for change;
  
  
• Insufficient resources;
  
  
• Overattention to the needs of one constituency (e.g., employer groups);
  
  
• Giving priority to short-term demands over long-term trends;
  
  
• Difficulties balancing competing goals and principles (e.g., equity vs. placement);
  
  
• Lack of formal procedures for change; and
  
  
• Ineffective leadership for reform.

        Greater experience with this model of local accountability will improve its usefulness as an analytic tool. For example, the model can serve as a basis for monitoring the effects of state and federal initiatives, such as the 1990 amendments to the Carl D. Perkins Act. A subsequent report based on this study will explore such policy applications.