Previous Next Title Page Contents NCRVE Home

EXECUTIVE SUMMARY:
Whose Performance Should Count

by David Stern

The 1998 Perkins Act requires states to establish performance measures for the purposes of gauging statewide progress in vocational and technical education and optimizing the federal investment in vocational and technical education activities (Section 113a). The law lists four core indicators of performance which states must report. However, the law does not specify exactly whose performance is to be measured. This leaves room for states to choose -- and their choices may have far-reaching consequences.

Two logical approaches

One approach would focus only on students who have had some significant degree of involvement in vocational and technical education. Since these are the students who are most affected by vocational programs, it does seem logical to hold vocational education accountable only for the performance of these students. Following this line of reasoning, the state could identify a subset of students -- for example, those who have taken three or more vocational classes -- and report only the attainments of these selected students. This approach may be especially appropriate at the postsecondary level, where a substantial proportion of students clearly specialize in vocational-technical studies.

At the secondary level, however, more than 95 percent of all students take at least one vocational course (Levesque et al., 1999, p. 33). Vocational education has become part of the high school curriculum , much like math or English, for the vast majority of students. This suggests a different approach: instead of measuring performance only for a certain subset of students,it would be equally logical to focus on certain kinds of attainmentthat vocational education is intended to promote. For example, a state could report what proportion of all high school seniors had reached certain vocational or technical skill proficiencies, and what proportion of all high school graduates are employed.

This possibility of focusing on vocational attainmentsat the secondary level has been overlooked, as many states appear to be focusing on vocational students (or "completers" or "concentrators") at both the secondary and postsecondary levels. This is unfortunate, because restricting performance measures to a subset of students may have the unintended and perverse effect of discouraging certain practices that are associated with better performance.

Vocational education, combined with an academic core, benefits the whole cross-section of high school students

Vocational education, in combination with academic instruction, has a powerful role to play in preparing high school students for both college and careers. The 1990 Perkins Act emphasized development of students' academic as well as vocational and technical skills, and integration of academic and vocational instruction. The 1998 renewal of the Perkins Act continued this emphasis.

In fact, nearly all students participate in some vocational education while in high school. A review of transcripts for a national sample of 1994 high school graduates found that 97.2 percent of public high school students had taken at least one vocational course, and 90.8 percent had completed at least one specific occupational course. (Specific occupational courses exclude keyboarding, work experience, family and consumer sciences, industrial arts or technology courses.) The average high school graduate in 1994 had completed four Carnegie units in vocational education courses. (Levesque et al., 1999, p. 33.)

Vocational, technical, and career-related courses are helping to prepare the entire cross-section of high school students. Many who take vocational courses go to college after graduating from high school. College-going students from the graduating class of 1982 accounted for 48 percent of all occupationally specific vocational coursework taken by members of that class during their four years of high school (Muraskin, 1993, p. 101). The number of students who combine vocational education with college-preparatory academic coursework has grown during the 1980s and 1990s. Among high school graduates who had taken three or more specific occupational courses, the proportion who also completed a set of core academic courses rose from 5 percent to 33 percent between 1982 and 1994 (Levesque et al., 1999, p. 40).

Several studies have found that high school students who combine a substantial academic curriculum with several vocational courses do better than students who omit either one of these two components. This was first noted by Kang and Bishop (1989), who discovered a positive interaction between the number of academic courses and the number of vocational courses in predicting post-high school earnings for males who did not attend college. Arum and Shavit (1995) identified students who had taken a set of advanced academic courses, a sequence of vocational courses, or both. Four years after senior year, individuals who had completed both advanced academic and vocational courses in high school had the greatest likelihood of being employed in professional, managerial, or skilled jobs, or being enrolled in postsecondary education. Outcomes for this group were better, or at least as good, as those who had taken advanced academic courses but no vocational education. Levesque et al. (1999), in addition to replicating these results, also found that students who combined a college-preparatory academic curriculum with a specific vocational sequence had gains in math, reading, and science test scores during high school that were similar to the gains of students who took only the college-prep curriculum -- and both of these groups gained substantially more than other students (p. 44).

Combining academic and vocational coursework also appears to benefit students who were not performing well at the beginning of high school. Specifically, several studies of career academies in Philadelphia, California, and elsewhere have found that low-performing students who entered academies improved their school performance and were more successful than comparison groups in the same schools. Career academy students had better attendance, completed more course credits, earned higher grades, and were more likely to graduate from high school (Stern, Dayton, & Raby, 1998).

The idea of combining vocational and academic coursework is central to High Schools That Work , a network of more than 800 high schools engaged in raising academic achievement by blending a rigorous academic curriculum with modern vocational studies (Bottoms & Presson, 1995). It is also a key component of the "New American High Schools" identified by the U.S. Department of Education: many of these schools are raising academic standards and expectations by structuring the curriculum around students' career related interests (Hudis and Visher 1999).

Accountability procedures under the Perkins Act could encourage developments such as these b recognizing the schoolwide contribution of vocational education. As suggested above, this would mean using measures such as the proportion of all high school seniors who attain certain levels of academic and vocational-technical skills; the proportion of entering ninth graders who eventually receive diplomas and/or credentials; and the proportion of high school graduates who go on to postsecondary education, employment, or military service.

These measures are not limited to a subset of students who are defined as vocational completers, concentrators, or specialists. Such a definition would have to be arbitrary and artificial, given the fact that almost all students take at least one vocational course. Identifying a group of high schoolers as vocational students would make no more sense than labeling some as math students, since virtually all students also take at least one math class. Although it would be possible to separate out students who took a relatively large number of vocational courses, this would reduce or eliminate the incentive for vocational educators and administrators to provide vocational courses for other students. This narrower approach to accountability could deny many students the benefit of combining vocational education with their academic coursework.

Evaluating and improving vocational education

The accountability strategy outlined here would provide measures of how well students are performing. Would it be possible to conclude from such data whether improvement (or lack of improvement) in students' performance was caused by their participation in vocational education? No, it would not. But narrowing the focus of accountability to include only a subset of students who took a large number of vocational courses would not answer this question, either. The best way to test whether participation in vocational-technical education improves students' performance would be to conduct a true experiment, in which some students are randomly assigned to vocational classes and others are not. The difference between the experimental and control groups would measure the impact of vocational education. This kind of study is seldom done because it is difficult and expensive to do. It would be unreasonable to expect states to conduct such research to fulfill their accountability requirement.

Without experimental studies using random assignment of students to programs, decision makers must rely on weaker kinds of evidence. For instance, several studies were cited above as evidence that students who combine vocational courses with a rigorous academic curriculum perform better than students who omit one of these two components. Yet these studies suffer from the well-known problem of selection bias. That is, students who take both a rigorous academic curriculum and a set of vocational courses may have more get-up-and-go to begin with. It may be their extra energy or persistence that accounts both for their extra course-taking and their subsequent success. Similarly, evaluations of career academies may also suffer from selection bias, if students who enroll in career academies are more motivated or competent than students in the comparison group. The problem of selection bias is pervasive in educational research, and conducting a true experiment is the best way to overcome it.

But there are other possibilities for gathering data that would inform decision-making, even in the absence of true experiments. One approach, which could yield practical benefits to students as well as good information for decision makers, would be to identify schools where student performance has been improving at a relatively rapid rate, and gather information on how vocational education has been structured and delivered there, in comparison to other places. This is a conventional kind of research procedure. If fast-improving schools are delivering vocational education in a different way than other schools, the difference in vocational education practices may be partly responsible for the faster improvement in student performance. And using schoolwide data avoids the problems of selection bias that occur when comparisons are made between different groups of students in a school. This approach could yield practical benefits to students as well as good information for decision makers.

Just finding that fast-improving schools deliver vocational education differently does not prove cause and effect. One other possible explanation is that faster growth in student performance is caused by other variables, but once the faster growth begins it permits schools to deliver vocational education differently. Another possibility is that both the faster growth in student performance and the difference in delivery of vocational education are caused by some other factor, such as new leadership.

To eliminate these other possibilities, the state could encourage other schools to replicate the vocational education practices found in the fast-improving schools. If the schools that adopt these practices experience a subsequent acceleration in student performance, this would be much stronger evidence that the practices were responsible for more rapid improvement. It would take several years to identify, replicate, and evaluate these promising vocational practices. But this is the kind of accountability strategy that would contribute to improvement of vocational education, in addition to satisfying federal mandates. And, again, it does not require anyone to identify a set of vocational students, completers, or concentrators.

What does the Perkins Act actually require?

The 1998 law requires states to report at least the following four performance indicators (section 113b):

"(i) Student attainment of challenging State established academic, and vocational and technical, skill proficiencies.

"(ii) Student attainment of a secondary school diploma or its recognized equivalent, a proficiency credential in conjunction with a secondary school diploma, or a postsecondary degree or credential.

"(iii) Placement in, retention in, and completion of, postsecondary education or advanced training, placement in military service, or placement or retention in employment.

"(iv) Student participation in and completion of vocational and technical education programs that lead to nontraditional training and employment."

Although the law explicitly says what to measure, it does not say that the first three indicators should be measured only for students participating in vocational and technical education. It permits states to define the first three indicators in terms of the whole student population -- for example, (i) the proportion of all high school seniors who attain certain levels of academic and vocational-technical skills; (ii) the proportion of entering ninth graders who eventually receive diplomas and/or credentials; and (iii) the proportion of high school graduates who go on to postsecondary education, employment, or military service.

Deciphering Congressional intent can be difficult, but in this case there are at least two reasons to conclude that Congress meant exactly what it wrote. First, parts of section 113 do refer explicitly to participants in vocational and technical programs only. The fourth core indicator, shown above, asks how many students participate in and complete vocational programs leading to nontraditional employment (defined as fields where one gender represents less than 25 percent). Also, section 113(d) requires states to submit annual reports which "shall include a quantifiable description of the progress special populations participating in vocational and technical education programs have made in meeting the State adjusted levels of performance...." (emphasis added). If Congress had wanted to use this phrase in specifying the first three core indicators, it presumably would have done just that.

A second reason to believe that Congress intended to let states define the first three core performance indicators in terms of the whole student population is that this continues a strategy adopted in the 1994 Improving America's Schools Act (IASA). For nearly 30 years, Title I of the Elementary and Secondary Education Act had distributed federal money to low-income schools for compensatory education, with accountability provisions stipulating that the money could only be spent on certain identified students. In addition to being cumbersome, the attempt to trace money to specific students led to those students being overtly labeled, segregated, and sometimes stigmatized. In 1994 this approach was replaced by IASA's new emphasis on schoolwide improvement, even though the federal money is still directed toward schools with high proportions of disadvantaged students. Title I of the Clinton Administration's proposed Educational Excellence for All Children Act of 1999 would continue this emphasis, and the accountability provisions in Title XI are designed to assist all students in meeting challenging state standards. If accountability in Title I, which is mainly concerned with a certain group of disadvantaged students, is no longer defined in terms of those students alone, then it makes even less sense to define accountability in terms of a subgroup of students for Perkins, which is concerned about certain kinds of learning and performance results.

Conclusion

Viewing vocational education as an integral part of the high school curriculum, like mathematics, recognizes that nearly all students are participating in vocational education. Designing an accountability system that encourages integration of vocational and academic coursework reinforces the trend toward keeping high school students' options open by helping them prepare for both work and further education. Conversely, separating high school students into subgroups for accountability could deny many students the benefit of combining vocational and academic studies. That would be inconsistent with the general intent of the Perkins Act, and with ongoing efforts in many states and localities to combine the strengths of academic and vocational education. In the long run, continued federal funding of vocational education in high schools is likely to be more secure if it is based not on a narrow subset of students but on a broad cross-section of the student population.

David Stern is the director of NCRVE and a professor of education at UC Berkeley .



References

Arum, R., & Shavit, Y. (1995). Secondary vocational education and the transition from school to work. Sociology of Education,68: 187-204.

Bottoms, G., & Presson, A. (1995). Improving high schools for career-bound youth. In W. N. Grubb, (Ed) Education through occupations New York, NY: Teachers College Press, 35-54

Hudis, P. M., & Visher, M. G. (1999, March draft). Aiming high: Strategies to promote high standards. Berkeley, CA: MPR Associates.

Kang, S., & Bishop, J. (1989). Vocational and academic education in high school: Complements or substitutes? Economics of Education Review, 8 (2), 133-148.

Levesque, K., Lauen, D., Teitelbaum, P., Alt, M., Librera, S., & Nelson, D. (1999, June draft). Vocational education in the United States: Toward the year 2000. Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Muraskin, L. D. (1993). Secondary vocational education: Availability, coursetaking, and outcomes. Rockville, MD: Westat, Inc.

Stern, D., Dayton, C., & Raby, M. (1998). Careeracademies and high school reform. Berkeley, CA: Career Academy Support Network, University of California.<http://casn.berkeley.edu/>


Previous Next Title Page Contents NCRVE Home