Formal program evaluation and outcomes assessment for Tech Prep has been limited, but when evaluations have been conducted they have tended to focus on compliance-oriented measures required by governmental units. Outcomes measures linked to enrollments, program completion, and job placement have been typical of the kinds of measures demanded by state and federal agencies. The national evaluation sponsored by the U.S. Department of Education concentrates much of its attention on having local Tech Prep coordinators estimate the number of students who reach specified points in the educational and employment system, such as high school completion, matriculation into two-year postsecondary education, two-year postsecondary completion, and job entry or matriculation into four-year postsecondary education. Such estimates may be useful in terms of understanding the potential scope and scale of the nation's emerging Tech Prep system, but they are not as helpful to understanding the way programs should operate and benefit students on more personal and consequential levels.
When local coordinators have been asked to specify outcomes they believe to be appropriate for students in vocational-technical programs, typically they identify educational, economic, and psychosocial outcomes (McCaslin, 1990). In 1992, Hammons surveyed local Tech Prep coordinators (educators), and found they supported a wide range of performance indicators for Tech Prep programs, including outcomes in all three of the categories described by McCaslin. Later, in 1994, many of Hammons' findings were supported when a national sample of local coordinators indicated a wide range of academic, vocational, and employment-related outcomes were a high priority for students (Bragg et al., 1994). Now, results of this concept mapping study show the three stakeholder groups of educators, students, and employers also give high priority to a wide array of student outcomes. All three stakeholder groups rate nearly all of 98 student outcomes statements at a moderate or high priority level. (For a summary comparison of how the three stakeholder groups rated the outcomes statements by clusters, see Appendix B.) This finding indicates it would be a mistake to limit assessments of student outcomes to only a few outcomes measures. Rather, multiple measures addressing a wide range of outcomes are necessary to determine how students benefit from educational and employment-related experiences. Unfortunately, assessment measures and methodologies are not available to conduct wide scale assessments of many of the student outcomes identified by the stakeholders in this study, heightening the importance of the need to create valid, reliable, and meaningful outcomes assessments for Tech Prep programs.
Are certain Tech Prep student outcomes grouped together in a logical, consistent pattern? Do the stakeholder groups perceive of the groupings (clusters) of Tech Prep student outcomes in similar and/or different ways? Are particular clusters of student outcomes more important than others to subgroups and to the group as a whole? By looking at the concept maps organized into nine-cluster solutions by the three stakeholder groups and the entire group of participants, it is possible to answer these important questions. Indeed, results show there are similarities in how the three groups conceptualize and prioritize Tech Prep student outcomes. All three stakeholder groups sorted many of the same student outcomes into three clusters labeled "Personal Attributes, Attitudes, and Employability Skills"; "School-to-Work Transition"; and "Work and Interpersonal Relationships." All of these clusters were given a rating near or at a high priority level of 4.0 (out of 5.0) by the subgroups, showing a high degree of consensus in how the stakeholders conceptualized these sets of student outcomes for Tech Prep (see Table 9). These results also suggest that student outcomes linking school-based education to effective attitudes and behaviors in the workplace are a high priority to all three stakeholder groups. Participants organized these outcomes into three distinct groupings: one dealing with personal attitudes and behaviors at work; a second focusing on interpersonal attitudes and behaviors at work; and a third concentrating on more transferable attitudes and behaviors between the school and work environments.
Three additional clusters were created by two of the subgroups. These clusters were "Information Use and Decision-Making," "Education and Career Attainment" and "Communications." Educators and employers were in agreement with respect to how they grouped and prioritized the first two of these three clusters of student outcomes; however, a notable exception was in "Education and Career Attainment" where the average cluster rating given by educators exceeded that of employers by a wide margin, 4.02 compared to 3.61. Although students did not link concepts associated with careers to educational attainment in the same way as educators and employers, the priority placed on the "educational attainment" cluster was also quite high (3.97). These results show outcomes associated with advancing within the educational system are a high priority to educators and students within the system, but not as much to employers outside of it. Consistently, employers give student outcomes associated with school-to-work transition and employment a higher priority than outcomes more closely associated with the educational system itself. This finding raises the question of what level of priority to place on educational attainment outcomes that address whether students are making progress on grade level, graduating from high school, matriculating to the two-year postsecondary level, and so forth. The question is particularly pertinent with regard to a program such as Tech Prep where a school-to-work focus is important to all stakeholders. Given that position, how far should Tech Prep programs go to accommodate the perspective of employers who know the workplace best? How much weight should be given to their preference for vocationally oriented outcomes over educational outcomes?
Yet, the issue is not simply one of educators and students giving greater priority than employers to educational outcomes. Indeed, the situation is much more complex. In fact, all the subgroups rated many outcomes statements associated with traditional academic subjects such as mathematics, science, English, humanities, social studies, and the fine arts lower than outcomes aligned with school-to-work transition and employment. Although all three groups organized outcomes statements into distinct clusters aligned with these academic concepts, most rated these clusters lower than the ones having a work or career orientation. Furthermore, within the clusters of "Math and Science" and "Analytic and Scientific," the stakeholders rated outcomes at the basic level more highly than those at the advanced level, showing a preference for students' mastery of more fundamental academic concepts over more advanced. Also, the academic clusters were segregated from vocational clusters on all the concept maps, with the academic concepts placed on the west side of the map and vocational concepts on the east. This result gives the impression that sets of outcomes associated with vocational and academic education may be both distinct and independent from one another. However, some clusters do not appear to fit this conclusion. In all of the concept maps, stakeholders created one or more clusters containing outcomes having to do with technology, information use, decision-making, work, and management. The outcomes within these clusters were drawn from across the disciplines such as humanities, social studies, science, and vocational-technical education. Typifying this kind of cluster is the one created by students labeled "Work, Technology, and Information Use" and one developed by employers labeled "technology and quality management." Within each of these clusters is the nucleus of outcomes taken from a wide range of vocational and academic subject matter, potentially providing ideas for integrating Tech Prep instruction.
An additional observation should be made about the nature of the three subgroups' conceptualizations of vocational and academic outcomes. Consistently, vocationally oriented outcomes received high or nearly high priority ratings, while academically oriented outcomes received lower (albeit not low but moderate) ratings. Within the clusters of academically oriented outcomes, statements linked to the academic areas of social studies and humanities received the lowest ratings. All three stakeholder groups created clusters with outcomes statements linked to the "Democratic Process and Career Awareness" or "Democratic and Participatory Strategies" and all three gave these clusters low mean ratings relative to the other clusters. In fact, the cluster labeled "Democratic and Participatory Strategies" created by employers rated the lowest of all clusters with an average rating of 2.99. Is this pattern a random occurrence or is there something about Tech Prep that suggests democratic outcomes should receive a lower priority than other outcomes? Public policy specifies that Tech Prep curriculum should be comprised of mathematics, science, English/ communications, and vocational-technical education. Rarely is the area of social studies or humanities mentioned as central. In addition, many local consortia and state agencies profess a primary purpose of Tech Prep is to "eliminate the general track." These constituents are attempting to improve education for the neglected majority of students who have been engaged in the general track, but in so doing may shift priorities away from some of the more traditional social and democratic functions of public education. Is this shift actually occurring? We could not identify data to suggest that such a curricular shift is occurring; however, it is an important issue to monitor. What are the consequences of shifting priorities away from traditional academic subjects to education more highly focused on school-to-work transition, technologies, and vocations? Without more attention paid to formal evaluation, this question will remain elusive.
In summary, this study attempted to better understand Tech Prep student outcomes from the perspectives of educators, students, and employers actively engaged in implementing Tech Prep. Knowing how these groups conceptualize student outcomes has important implications for understanding the fundamental objectives of Tech Prep, for planning and implementing programs, and for assessing outcomes in the future. Also, by uncovering various conceptualizations of Tech Prep, it may be possible to identify conflicting perspectives held by disparate stakeholder groups. Having information from still more stakeholder groups from other localities such as rural and suburban areas would help to illuminate the ways other constituents think about Tech Prep. In addition, obtaining information from policymakers, parents, counselors, and still other groups could help in the development of outcomes assessments. As Tech Prep implementation continues, more attention must be devoted to student outcomes, and this study takes an important next step in that direction.