Crain, R. L., Allen, A., Thaler, R., Sullivan, D., Zellman, G., Little, J. W., and Quigley, D. D. (1999). The Effects of Academic Career Magnet Education on High Schools and Their Graduates (MDS-779). Berkeley: National Center for Research in Vocational Education, University of California.

CHAPTER  3
The Academic Effects of Career Magnets

• Methods and Data
• Findings
• Discussion and Conclusions
• References

      Most American high schools have long been structured on the premise that academics and vocational education should be kept separate. That model is rapidly breaking down. With skilled blue-collar jobs moving overseas and the service sector growing, working class jobs require higher levels of literacy. A new movement is endorsing that--even for the college-bound--the right approach is "contextualized learning"--that is, the integration of career and academic preparation (Berryman & Bailey, 1992; Resnick, 1987). Others argue that the high school curriculum has no time in its college preparatory track for non-academics--that college-bound students are losing the achievement race with Europe and Asia and that even students not destined for college need higher levels of literacy and numeracy in order to work in the service sector. The issue is time--the school day is a pie of a given size, and reallocating a larger portion to careers cuts into academics. These critics have a point: The great improvement in cognitive performance of the American population over the last century has been largely the result of increased school attendance and, hence, more years of "time on task."

      Proponents of school-to-work programs in high school argue that there are "slack resources" in school because adolescents are not motivated by traditional academic programs and do not spend much of their time learning; anything that shows students a connection between academic learning and their future will increase their interest in school and take time away from television and sleeping in class. It seems unlikely that this debate has any simple answer. It is more likely that there are some ways in which a school-to-work program encourages higher academic performance, and others in which it is harmful.

      We have been provided a unique opportunity to measure the impact of academic career integration on academics. We have test score data along with descriptions of the programs that students applied to for almost eight thousand students who were randomly divided into career magnet and regular comprehensive high school programs. This is the largest test ever done of an educational program using random assignment.

Methods and Data

      Administrators were asked to base their responses only on the students in their academic career magnet program, excluding students in other academic career magnet programs in the same building or in the school's regular comprehensive program, and students in any ESL/bilingual or special education programs. Of the 59 programs for which we had acceptable numbers for analyzing the experiments, only 49 were surveyed. One program had closed, and nine program heads refused to be interviewed.

      One goal of the survey was to determine how successful academic career magnets have been in integrating academic and vocational education. We concluded that programs varied greatly not only in the particular type of academic career focus chosen but also in the extent to which they had implemented it.

      The minimum requirement for graduation is passing regular reading and math examinations, or one or more advanced versions of those examinations. Our data file contains these scores and also SAT Verbal and Math examination results. Students, either at their own discretion or that of their counselor, may take these examinations in any of the years covered by our Student Records File. As a result, the number of students in any one of the three reading levels taking a particular examination in a specific year may be small, even though we have data on 7,987 students who applied for admission to 49 programs in the Fall of 1988.

      Step A: The first is a zero-order Pearson correlation, computed across the 49 programs between the extent of a career program component and the "program effect," and the performance of applicants to the program who were lottery winners compared to the performance of lottery losers among the applicants, both adjusted for seventh- and eighth-grade academic performance. This is the "perfect" experimental result in that it is unbiased. Being unbiased, it includes students who were randomly selected into the "experiment" and "control" groups but did not actually experience the "experiment" or "control" treatments. As already discussed, some lottery winners (29% for whom we have student outcome data) did not attend their first choice academic career magnet program, and some of the lottery losers (18.3%) received the experimental treatment because they went to the academic career magnet. This raises the possibility, admittedly slim, that a significant correlation is the result of a difference between the so-called "experimental" and "control" subjects who, in fact, were not actually in the treatment and control groups (i.e., we might have students who won the lottery and went to a highly selective school or to their comprehensive school and who strongly outperformed students who lost the lottery but actually went to the academic career magnet program!). Even so, a significant correlation between a program having a particular characteristic and the program having a high or low examination score is the strongest possible evidence that the relationship is present in these programs.

      Step B: To estimate the actual magnitude of the effect (and as a check for the possibility of the assignment errors discussed above creating a false positive finding), we computed mean test scores from the individual data file, comparing the individuals who won and lost the lottery, but separating those winners who did not go to the program and also those losers who did go. These tables are no longer an unbiased experiment, but they provide a measure of the size of the significant program-level correlations found in Step A.

      Procedure for Step A: Since each career magnet program provided up to three separate randomized experiments (one for each seventh-grade reading level), we were able to construct outcome measures comparing the randomly admitted students (including those that did not go to the career magnet) to the randomly rejected students (including those who did go to this or another career magnet program). We correlated the presence or absence or magnitude of particular program characteristics as reported in the Program Administrator Survey with each program's value of (i.e., mean performance of lottery winners minus mean performance of lottery losers) from the Student Records File. Since the true number of degrees of freedom should be based on 49 (the number of programs), and not 7,987 (the number of students), we aggregated the data to the program level.[3] The apparent effect of each program was measured by computing a pretest-adjusted score, adjusted by regression for seventh- and eighth-grade standardized reading and math scores, grades, and absences, for each student outcome. The mean for all students who applied to each program as their first choice and lost the lottery was subtracted from the mean for all first choice winners to the same programs.

Findings

      Academic career magnet program students do not have higher or lower reading scores, take advanced graduation examinations more or less often, or have higher or lower absenteeism. They do seem to have slightly lower math scores. Proponents of school-to-work will be disappointed by this, since many of them have been arguing that adding an academic career focus should enhance academics by increasing student motivation and integrating academics and careers. Advocates of choice will also regret these results since they expected the schools to perform better simply because the free market should have weeded out the academically weaker programs. Also, many social scientists would have predicted that students choosing these programs would gain a sense of ownership that would translate into higher motivation. On the other hand, many advocates of school reform who have had their expectations tempered by program failures in the past will be reassured that the career magnets were able to introduce the academic career focus and all its attendant work on student adolescent development without examination scores declining.

      We located two factors with quite powerful effects on academic achievement in some of these programs: (1) bringing the workplace into the school and (2) bringing the school into the workplace.

Computer Usage
      One way that academic career magnets bring the workplace into the school is by importing the technology of the workplace and its culture. Computers are becoming ever more ubiquitous in every occupational field. Whether in an office or on the shop floor, employment requires knowledge of and experience with computers. As a result, computers are not only part of the technology of production in the workplace, they are also increasingly part of the culture of the workplace. In our 1991 survey, 90% of the program administrators indicated that their students used computers either in classes or in a computer lab; however, the number of computers available in each program varied greatly, as did the regularity and extent of their usage by students in the program. A variable, Computer Usage, was constructed from five questions asked of program administrators: Do students in the program use computers? If yes, how many computers are available to students? What proportion of students use computers in a typical week? How many hours per week do students at each grade level use computers? To what extent do students in this program use computers compared with the rest of the school?

      Figure 3.1 shows a plot of the effects of 34 academic career magnet programs (those which had sufficient cases for analysis) on the standardized mean difference in 1990 Regular Math Exams (net of seventh- and eighth-grade test differences) against the extent of computer usage in that program. The horizontal axis is the extent of student computer usage from a low of zero to a high of seven on the questionnaire scale. The vertical axis is the mean score of all lottery winners applying to the program minus the mean score of all lottery losers applying to the program.

      The first thing to notice in the plot is that the "program effects"--the performance of a program's lottery winning applicants minus the mean score of the lottery losing applicants--are as often negative as positive. Since the "n"s are small in some of these experiments, there is considerable sampling error, but it is still interesting that there are 11 programs in which lottery losers outperform lottery winners by over .5 standard deviations, and only 6 programs in which lottery winners outperform lottery losers by this much. (This pattern appears on other tests of both reading and math.) Even more interesting, however, is the size of the correlation between computer usage and the program effect on 1990 Regular Math Exam pretest-adjusted scores. All seven of the programs with high computer usage (scores over 5) have neutral or positive program effects, but five of six programs with low computer usage (below 3) have clearly negative program effects.

      The plot and correlation are impressive. From the plot, we can estimate that the effect of winning lottery admission to a low-computer usage academic career magnet program (programs ranked below 4 on our scale of computer usage) is to lower 1990 Regular Math Exam pretest-adjusted scores by about five-tenths of a standard deviation (about 50 SAT points) below what would be expected of similar students in other high schools, and the effect of being in a high computer usage academic career magnet program (programs which ranked 4 or higher on our scale) is to raise the 1990 Regular Math Exam pretest-adjusted scores by about one-tenth of a standard deviation. When we look only at the seven programs with the highest computer usage, we see a stronger positive effect of winning the lottery, on the order of one-quarter of a standard deviation. Since some lottery losers did enter academic career magnets by school selection, the difference between career magnet programs and comprehensive schools is attenuated, and the apparent difference between the impact of high-lottery and low-lottery programs is an underestimation. In Appendix B, we estimate that Figure 3.1 and other figures like it underestimate the effect of computer usage (or any other program attribute) by about one-half of a standard deviation.

      Although we are primarily concerned in this section with the contribution of computer use to raising examination scores or preventing them from falling, the most important finding in the figure is the lower scores of most career magnets. We think the most likely explanations are the same as we made for the low graduation rates in many career magnets: (1) that career magnets ignore many of their weaker students, and (2) that comprehensive schools are pressed to keep scores up, but career magnets are not because the necessary data is not made available.

      The next analyses confirm that computer use holds examination scores up. We present a number of tables displaying large and statistically significant findings, supported by other tables that sometimes have large but not statistically significant findings. We have relied heavily on program-level data, since it is the most accurate way to measure the effects of the experiment, but this usually means that we have data from only 34 programs, so only extremely large findings, such as the relationship in Figure 3.1, will be statistically significant. Even some of the individual-level analyses have as few as 100 cases because we are selecting certain subpopulations to make various points.

      Table 3.1 presents our second way of looking at the data. This table is based on individual data for the 1,470 average reading-level students who took the Regular Math Exam in 1990, and compares students who applied to programs that (1) had high or low levels of computer usage, (2) either won or lost the lottery for admission, and (3) did or did not attend the career magnet to which they applied. Looking at the first column, we see that students who applied to programs that do not use computers very much (this includes a number of pre-law and more academic programs) are the sort of students who score slightly better than expected on the 1990 Regular Math Exam. The bottom row shows that those who lost the lottery and did not attend the career magnet program (mostly because they were not admitted) scored about .06 above the average student in the test pool, adjusted for seventh- and eighth-grade examination scores. Those who lost the lottery and attended the career magnet (these were all school-selected) should be above average in talent and their small positive gain in math achievement should not surprise us. Those who won the lottery but elected not to go to the career magnet school tend to show scores typical for students with similar pretest scores (.03 standard deviations below average). But students who won the lottery and enrolled in the academic career magnet programs with low computer usage tended to score quite badly, almost a quarter of a standard deviation below expectations, and even lower when compared to lottery losers. In fact, they performed below lottery losers who did not attend their academic career program by school selection by [(-.23) - (+.06)] = .29 of a standard deviation. The second column tells a different story for students who applied to high-computer-usage programs. The fourth row shows that they tend to be average students since those who lost the lottery and did not attend an academic career program have only average scores, and the third row shows that those students who lost the lottery and were school-selected into the high-computer-usage programs also have typical scores (-.01 and -.03, respectively). Students from this group who won the lottery and attended an academic career magnet performed slightly above average (+.07). Students who won the lottery but selected a different school also did slightly better than expected, perhaps because they had the academic talent to be offered seats in highly selective career magnets or other career magnets with strong mathematics programs.

      Our best estimate of the effect of high computer usage in these academic career magnet programs is based on the third column of the table, which shows the differences between the data in columns 1 and 2. The difference in the bottom row is between lottery losers who did not attend their career magnet choice, mostly because they were not admitted; the difference is slightly negative, dLN = -.07, suggesting that the pool of applicants to the high-computer-usage programs was slightly weaker than the pool of applicants applying to the low-computer-usage programs. But among the students who won the lottery and chose to attend the career magnet school, there was a large difference favoring the high-computer-usage programs: dWG = (+.06) - (-.23) = .30 s.d. Our best estimate of the effect of the career magnet is D, defined as D = the difference between the performance of students who entered high-computer-usage programs compared to those entering low-computer-usage programs, dWG, adjusted for the apparent selection bias, dLN: D = dWG - dLN = .30 - (-.07) = .37 standard deviations.

      The estimate is consistent with the estimate shown in Figure 3.1. Although the results are not unbiased (because they pool students from different experiments), it is also not as attenuated as the plot in Figure 3.1 is. While the "n"s in Table 3.1 are small, the results are consistent with the plot and seem plausible.[4] Figure 3.1 and Table 3.1 indicate that heavy use of computers will offset the considerable loss to be expected in career magnets, which are usually focused on non-academic careers. Both show the largest effect to be a negative one for students attending career magnet programs with low computer usage. The -.23 in Table 3.1 is the largest effect, and looks similar in size to the average of the effects of the career magnets with low computer usage in Figure 3.1.

      Table 3.2 shows a slightly weaker relationship for 1991 test results, consistent with the program-level correlation in 1991 of .366 (p = .046). Figures presented are again standardized adjusted scores and show that lottery winners in academic career programs with high computer usage have scores one-fifth of a standard deviation (dWG = .20) above those of winners attending programs with low computer usage. Whereas in 1990, the benefits of higher computer usage resulted from a small gain in achievement in high-computer-usage programs coupled with a large loss in low-computer-usage programs, in 1991 there was no loss in the low-computer-usage programs and a large gain in high-computer-usage programs.

      At first, it was thought that programs with high computer usage might be using computers to teach math; however, in visiting programs, we found little evidence of this. There is possibly something about learning how computers function, such as logic, which is transferable to math, but we have no way of studying this hypothesis with our data. However, we were able to test a third hypothesis--that computers improve student motivation.

      All of the data we are using on our individual file come from the school district's official records for each student. This file contains no data on class-cutting; however, it does contain a record of which students were absent when they were scheduled for a standardized reading or math examination. For each program, we computed the extent to which lottery winners were absent from these examinations more or less often than were students in the control group, adjusting for any random seventh- and eighth-grade background differences. We then correlated this measure of program effectiveness in reducing examination-cutting with each program characteristic, using reduced examination-cutting as a measure of increased motivation. Students in high-computer-usage programs were less likely to be absent when standardized reading and math examinations were given.

      Table 3.3 shows the proportion of students present for whichever standardized reading examination they had signed up to take in 1991 by the level of computer usage in their career magnet program. The aggregate file correlation was .304 (p = .081). Table 3.3 shows that students in programs characterized by high computer usage are 9% (p < .05) less likely to be absent for a standardized reading examination (dwg) than are students who won the lottery to low-computer-usage programs. Note that dLN = 3%; this difference among lottery losers favoring applicants to high-computer-usage programs suggests a selection bias, that students who apply to high-computer-usage programs will be less likely to miss a reading examination no matter what school they attend. We estimate that, corrected for selection bias, the true program effect is D = 9% - 3% = 6%. Since absenteeism falls between 18% and 25% on these examinations, 6% represents a one-quarter reduction in absences.

      Table 3.4 is similar to Table 3.3 except that it concerns the proportion of students present at standardized math examinations that year. Based on an aggregate file correlation of .367 (p = .036), it shows that lottery winners to programs high in computer usage are 8% less likely to be absent from a standardized math examination than are students who won the lottery to programs low in computer usage. This is a one-third reduction in absences, and, in this case, there is evidence of a small selection bias against the career magnets, so our estimated effect is larger: D = .08 - (-.01) = .09 (p < .05). These findings support the hypothesis that increased use of computers enhances student motivation, and we assume that students with greater motivation pay more attention in class, study harder, and try to do better on their examinations.

      However, we must test for the counter hypothesis that the correlation between increased computer usage and higher math scores or attendance on standardized examinations is the result of creaming by programs with high computer usage. By "creaming," we mean the artificial inflation of a program's performance by (1) discouraging weak students from taking the examination, (2) delaying students taking any of the examinations, or (3) encouraging strong students to take the examinations. To check this, we did an analysis of which students took the Advanced Math Exams in 1991. If higher scores on this examination are the result of creaming, one would expect to find one or more of the following scenarios: (1) The proportion of students taking an Advanced Math Exam in high-computer-usage programs is low compared to students in programs with low computer usage; (2) the proportion of students taking a second Advanced Math Exam is low; (3) the proportion of students taking a first Advanced Math Exam should be low; (4) the proportion of students who have taken one Advanced Math Exam, but are not taking one this year, should be low; and (5) the proportion of students who have never taken the Advanced Math Exam and are not taking it this year should be high. We found no evidence of creaming.

      Table 3.5 contains data on the four different possible test-taking patterns: the proportion of students in low- and high-computer-usage programs who are (1) taking their second Advanced Math Exam (some are retaking, but there are three separate math examinations so they may have passed different examinations in 1989 or 1990); (2) taking their first Advanced Math Exam; (3) not taking an examination this year after having taken one in the past (some of these students failed an Advanced Math Exam and are now taking the easier Regular Math Exam); and (4) the proportion who were not taking, and had never taken, an Advanced Math Exam. The top half of the table shows data in each of these four categories for students who won the lottery and chose to attend their first-choice career magnet school. (By definition, they must total 100%.) The bottom half of the table shows data for students who lost the lottery and did not attend their first-choice career magnet program. For brevity, we have omitted the two middle rows shown in Tables 3.1 through 3.4: the lottery winners who did not attend the career magnet, and the lottery losers who did attend the career magnet. The third column of the table, thus, only shows the two differences, dWG and dLN, defined in the discussion of Table 3.1. Computing D = dwg - dln, we see that the apparent effect of attending a high- as opposed to a low-computer-usage program is to increase the chances of a student taking a second Advanced Math Exam in 1991 by 3.4%, increase the chances they will take their first examination this year by .9%; increase the chance that they had taken a regents examination earlier but were not taking one this year by 2.5%, and decrease the chance that they had not ever taken an Advanced Math Exam by the time they finished their third year of high school by 6.8%. In other words, 19.1% of lottery winners in low-computer-usage programs took advanced examinations while 26.5% in high-computer-usage programs took advanced examinations (p < .05). This is not because of some innate difference between the two groups in desire to take the examinations; applicants to high-computer-usage-programs who did not go to a high-computer career magnet are .6% less likely to take the Advanced Math Exam. This means the higher Advanced Math Exam scores shown for high-computer-usage programs in Tables 3.1 and 3.2 are despite the fact that the test-taking pool is one-third larger in high-computer-usage programs.

      In addition to having higher math scores, average students who won the lottery to attend programs characterized by high computer usage were more likely to satisfy graduation requirements in reading than were lottery winners who went to programs characterized by low computer usage. The aggregate file correlation was .308 (p = .073). Students reading below grade level in seventh grade who won the lottery to attend programs characterized by high computer usage were found to have taken higher level reading examinations in 1991 and higher level math examinations in 1992 than similar students who won the lottery to programs low in computer usage (no table shown).

      No significant correlations were found for students reading above grade level in seventh grade who won the lottery to programs high in computer usage. Since scores on standardized examinations are usually correlated with SES, we think that these students, regardless of the program to which they were admitted, are more likely to have access to computers outside of school. Thus, an increased use of computers in school might not have a significant effect on measures covered by this research. It may also be that these students are already motivated by their high grades and do not need the motivational boost from computers.

Specialized Career-Related Equipment
      Our survey of program administrators asked whether students used other types of specialized equipment besides computers. Specifically, we asked whether the program had other types of equipment such as business machines or medical instruments that could be used to promote student career socialization. Among programs surveyed, half indicated that students did use such specialized equipment; however, there were basically no meaningful correlations of outcome variables with our scale of special equipment usage on the aggregate file. The use of special career-oriented equipment in this data set appears to have no effect, either positive or negative, on academic performance.

Computers and Specialized Career-Related Equipment
      A variable was created to identify programs that used both computers and other specialized career-related equipment. Programs characterized as high in both computer and specialized career-related equipment usage correlated significantly with taking the Basic Reading Exam in 1990 (.522, p = .007); students should have taken that examination the year before. In addition, average students who won the lottery to attend programs characterized by high computer and special career-related equipment usage scored poorly on the minimum-for-graduation Regular Reading Exam in 1991 (-.379, p = .060) and generally failed to pass the Advanced Math Exam that same year (-.411, p = .055). They were, however, more likely to be present for whichever math examination they were scheduled to take that year (.442, p = .029). We think that programs characterized as high in computer and specialized career-related equipment are more career oriented than are programs that are high in computer usage but have no other specialized equipment. As will be discussed later in this report, beyond a certain point, concentrating on career preparation seems to reduce academic achievement.

Independent Projects
      While computers and other special equipment are mechanisms for bringing both the technology and culture of the workplace into the school, independent and team projects can do this as well. Independent projects are naturally quite common in high schools for students who are expected to go on to college. Students receive term paper and science project assignments that enculturate the student into the behavior expected of them in college. What is different in this case is that in academic career magnets, such projects are also socializing students into the culture of the workplace.

      Most program administrators reported that students had opportunities to do independent projects, and that the number of students taking advantage of those opportunities increases with grade level as does the number of hours per week students are expected to work on such projects. At present, we have no evidence that independent projects either help or detract from the academic achievements of most students. We found no pattern of significant correlations between programs ranked high on our scale for independent projects (based primarily on the number of hours students were expected to devote to such projects) and student outcomes on the aggregate file. While for this data set there is no evidence that the use of independent projects leads to an improvement in student academic performance or motivation, there is also no evidence that such projects result in lower academic achievement.

Teamwork
      Another characteristic of academic career magnets that brings the workplace into the school is teamwork. Career magnet programs are as likely to stress team projects as independent projects. Apparently this is done to help students develop interpersonal skills that presumably will be useful on the job after graduation. A scale of teamwork was developed based on the extent, according to program administrators, that students are involved in team projects as opposed to individual projects. We found no significant pattern of correlations, either positive or negative, between teamwork and student outcomes.

Job Placement Service
      It is reasonable to assume that when students see others graduating and getting jobs based on the career training they have received, they perceive the career preparation aspect of their curriculum to be much more relevant to their lives. Program administrators were asked two questions about job placement: (1) Does their program help locate employment for graduating students, and (2) Approximately how many are actually placed in employment through the program? Thirty-four of 61 programs surveyed provided job placement services. Of these 34 programs, only 19 were able to tell us how many students were placed, and their responses ranged from two to 70. For these 19 programs, we computed a ratio of the number of students placed to the total number of students graduating from the program. Programs that did not provide placement were coded as zero. (The 15 programs that provided placement services but could not estimate the number placed were omitted.)

      When student outcomes in the aggregate file were correlated with our scale of program placement, we found evidence that more job placement services weakened the academic performance of the program. In 1989, few students in such programs who were reading on grade level in seventh grade took the Basic Reading Exam (r = - .793, p = .000), and those that did performed poorly (r = -.350, p = .042). In 1991, students in programs with high placement had high scores on their Advanced Math Exam (r = .331, p = .073) and on the Math SAT Exam that same year (r = .442, p = .085); however, this was evidently the result of creaming. For students from this reading level cohort, there were negative correlations of placement with two variables designed to determine the difficulty of the standardized reading and math examinations taken each year. Though such correlations were never significant, the fact that they were always negative indicates that students in programs offering a high level of placement services were not being motivated to do more than the minimum required for graduation and were not preparing for college.[5]

      That the greater proportion of students in programs ranked high on our scale of placement services are evidently not preparing for college should not be taken as an indication of reduced motivation among those who took any of the standardized reading examinations for graduation. We found a correlation of .566 (p = .000) between our program placement scale and the proportion of students present for reading examinations in 1992. After removing lottery winners who failed to attend and lottery losers who were program selected, winners in programs that ranked high on our job placement scale, after adjusting for pretest variance, were almost 10% more likely to be present for this standardized reading examination than similar students in programs ranked low on that scale. The two-way interaction effect was significant at p = .019.

      To provide a functional job placement service for its graduates, career magnet programs must develop a relationship with potential employers. This takes time and a lot of work on the part of one or more members of the program staff. To maintain that relationship, the graduates who are placed must be qualified. A lot of time and effort would be lost if the programs were to place unqualified students with an employer with whom they had cultivated a relationship.

Academic Career Focus
      In the Program Administrator Survey, respondents were asked if their program was (1) college preparatory, (2) college preparatory with an emphasis on a career in a specified field, or (3) career preparatory. One-quarter said they were college preparatory, one-fifth said they were career preparatory, and slightly over one-half said they were both. They were then asked whether their program prepared students to work in a particular career field, and whether their program provided graduates with a special employment certificate or license. The answers to these three questions were combined to create a Career-Focus Scale. It was found that students in academic career-focused programs took their examinations later in their high school career than other students in our data set. For students who were reading on grade level in seventh grade, this scale correlated at .305 (p = .080) with taking the Regular Math Exam in 1990 and at .375 (p = .029) with taking the Regular Reading Exam in 1992. In 1991, the scale of academic career focus had a negative correlation with taking the Verbal SAT (r = -.303, p = .082) as well as a negative correlation with our level-of-test variable, indicating that students in programs ranked high on academic career focus tended to take a lower-level standardized reading examination.

      In terms of standardized math examinations, average students in programs ranked high in academic career focus again tended not to take the Math SAT Exam. In 1991, the correlation between taking the Math SAT Exam and academic career focus was negative, though not significant. In 1992, the correlation for these same two variables was -.290 (p = .096). This tendency towards achieving only the minimum requirements for graduation should not be taken as an indication of low student motivation, however. As with job placement services provided, we found a correlation of .289 (p = .097) between our Career Focus Scale and the proportion of students present for reading examinations in 1992.

      While it is true that students must pass both of these examinations to graduate, and we would expect this cohort of students to graduate in 1992, it is also true that both regular examinations are the minimum required for graduation. Both can be substituted for by taking, and passing, a corresponding advanced examination. Taking advanced examinations means going beyond the minimum for graduation and might be viewed as testing one's preparation for going on to college or at least attempting to maintain the option of going to college. Students in programs characterized by a high academic career focus tend to take only the minimum required examinations and not to take either SAT Exam, an indication that these programs are focusing students on employment after graduation at the expense of college.

Previous Work Experience of Faculty
      Fifty-one of the 61 administrators responding to our Program Administrator Survey reported that at least some of the teachers in their program had worked in the field for which they were preparing students. Twenty-five programs reported that most of their instructors had such work experience. The two independent variables, Career Focus and Number of Faculty with Previous Work Experience, are significantly correlated at .47 (p < .01). For students who were reading on grade level in seventh grade, Faculty with Previous Work Experience was found to correlate with fewer students taking the Math SAT or the Verbal SAT Exams in 1992. The aggregate file correlations were -.344 (p = .047) for Math and -.300 (p = .077) for Reading. Table 3.6 is a summary of eight individual-level differences, similar to Tables 3.1 through 3.3, but reports only the best estimates, "D" (defined in the discussion of Table 3.1 earlier) of the effect of attending a program in which more faculty have work experience. This table shows that, from the individual student file, increasingly fewer students in programs ranked high on our scale of Faculty with Previous Work Experience were taking either the Advanced Math or Math SAT Exams. Instead, the students in these programs were more likely to take the Advanced Reading Exam in 1991 and 1992 instead. By postponing the advanced examinations until their last two years of high school, and not taking the SATs, they seem to be indicating that they will not apply to any even moderately selective four-year colleges. As already stated, however, it is only the failure to take SAT Exams in 1992 that was significant on the aggregate file.

      Academic career magnet programs that are more strongly oriented towards a career than is the average for this sample of programs seem to have a definite negative influence on the development of student plans for going on to college. These findings would seem to correspond to and support those found under the section "Career Focus," namely that programs that direct students' attention out of the school towards a career will emphasize minimum graduation requirements. Advanced Exams, and to a greater extent SAT Exams, emphasize schooling and at least the possibility of going on to college. In the dichotomy between purely academic and purely vocational high school programs, clearly those with a high academic career focus, with a high proportion of teachers having had previous work experience in the field for which they are preparing students, are going to be more like the vocational model than the academic.

School Activities
      The Program Administrator Survey included a number of items that seem to fall in an intermediate zone between bringing the workplace into the school and bringing the school into the workplace. These were classified as career-related extracurricular activities and included trips to work sites, mentors, outside lecturers, career-related clubs, and workshops on résumé writing and/or interview skills. While these variables correlated among themselves, the only other variable from the Program Administrator Survey with which they correlated was the Job Placement Services variable. When included as independent variables on the aggregate file, there were no significant correlations between these career-related extracurricular variables and student academic outcomes.

      The reason for this may be that so few students actively participate in any of these activities that at the aggregate level their effect, if any, cannot be detected. McNeil (1995) analyzed the "High School and Beyond" data set and found no effect of vocational clubs on graduation rate once ethnicity, gender, age, SES, one-parent household, test scores, and academic and vocational technical placement were controlled. His analysis was at the individual level, looking just at those students who acknowledged having participated in such clubs. We are unable to distinguish between students who have and have not participated in career-related extracurricular activities with our data.

Internships
      Internships and part-time work assignments provide an opportunity for "real-world" experience, building a bridge between school and work. Moreover, they provide students with the opportunity to explore and test themselves in their chosen field while they still have time to alter their educational and career plans.

      Five out of every six program administrators responding to our survey indicated that at least some of their students do some sort of internship. They indicated that internships were not used in the ninth and tenth grades as much as in the eleventh and twelfth grades. Only nine of the 61 respondents said their school had no internship program while more than one-third indicated that students were required to participate in internships. In addition, about 60% of the respondents indicated that program staff located internships for students.

      Among students who were average readers in seventh grade, there was a correlation of .419 (p = .014) between internships and the proportion of students in attendance for standardized reading examinations in 1989 and, again, for students reading above grade level in 1990 (.472, p = .077). Since internships are undertaken during a student's third or fourth year of high school, the increased examination presence evidenced in this table may simply be an indication that programs stressing internships tend to push students to appear for scheduled examinations in order to complete graduation requirements on time. It may also be a motivational effect similar to what we think happens in programs with high computer usage: Students are eager to meet requirements so they can qualify for internships. While we were unable to find any evidence among typical students that internships improve academic performance, we also found no evidence that internships lower these students' academic performance or motivation.

      Among students who were reading below grade level in seventh grade, there was a significant correlation (.334, p = .058) between internships and Regular Math Exam scores in the students' second year of high school. For students from this reading level, this examination is not usually taken until the third or fourth year of high school. That same year, 1990, there was also a significant correlation for these students between internships and the number of credits earned (.323, p = .048). This again seems to indicate that students in programs stressing internships are pushed to complete graduation requirements. The following year, 1991, students from this reading level did well on their Regular Reading Exams, having an almost significant correlation between that examination and internships of .330 (p = .052). As would be expected, however, students from this group earned fewer credits in 1991 than similar students in programs that do not stress internships (r = -.315, p = .044).

Program Unity
      A scale of program unity (also referred to as program isolation) was constructed from a series of questions asked of program administrators. These included whether or not program students took classes with other students in the school, whether program students had special classes that other students in the school did not take, how many special classes were taken at each grade level, and whether or not students in the program had their own counselors.

      Since the number of special classes taken by students in career magnet programs increases with grade level, students probably need to complete many of their graduation requirements early. We found no evidence that they were passing their comprehensive examinations early, however; there was no interpretable pattern of significant correlations between program unity and any dependent variables.

Discussion and Conclusions

      One reason the computer seems to be so universally effective is that it uses a straightforward pedagogy--individualized instruction, moving at a pace appropriate to the student, and providing almost instantaneous feedback and rewards--that makes it a powerful teaching tool. It is easy to understand why students would be motivated by spending time doing any of a variety of tasks with the computer. One school we visited talked about using computers as a remedial technique to help students who had difficulty learning accounting from the textbook. Students needing additional help were given a class in computer-based accounting that succeeded in bringing students up to the level they needed for more advanced accounting classes.

      We do not think that the improvement of math performance in those career magnet schools with a large amount of computer usage is a direct effect of teaching mathematics via computer. In visiting schools, we concluded that the computers were more often used in career-related classes, such as accounting, to teach secretarial or programming skills. There may be some transference cognitively from computer programming to the algebra that appears on standardized tests, but it is more likely that the transference, if it occurs at all, is in the level of symbol use. The use of icons and keystrokes to represent operations and the process of locating or moving numbers and text according to rules may make mathematical operations seem more familiar and easier to learn.

      Our data supports the hypothesis that computers provide a large motivational boost. Students who can master computers may be more confident about their abilities to deal with mathematics. It may also be that simply the pleasure of working with computers makes the drudgery of a regular academic class lighter. It is possible that gaining computer knowledge gives students self-confidence about their postgraduate success, increases their confidence about their ability to get a good job or succeed in college, and persuades them that mathematics has more relevance than they thought for the work they will do after graduation. It is impossible, with the data file we are using, to empirically test these hypotheses, although they are generally supported by the opinions of the school staff members we have interviewed.

      It is an oversimplification to say that mathematics performance is lower in schools that stress placing students in positions after high school because career education and academics are in competition. Exactly how do they compete? One possibility is that students are not encouraged to take advanced mathematics because of the need to offer them career education courses. Another possibility is that the program administration may be indifferent to the mathematics performance of their students so that mathematics teachers are not motivated to push their students harder. Passing the Regular Math Exam is required for graduation, but in high schools accustomed to high graduation rates, the mathematics department may not be under pressure to try to get every student to pass the examination. We also hypothesize that programs focused on placing students in employment will tend to be highly selective, looking only for those students who can bring credit to the program, and hoping, either consciously or unconsciously, that other students will leave the program. In some programs, only a minority of students are permitted to stay in the program; the others are sent into the comprehensive high school where the program is located or forced to transfer to the high school in their home neighborhood. In addition, since the program is imbedded in a larger school, there is no reporting requirement that the program be informed about the performance of its students overall. Thus, the schools have little incentive and little information to motivate students to improve their mathematics performance.

      Most school districts do not have an accounting system to detect the sorts of successes and failures we have uncovered. Most public school systems normally measure the performance of a school by looking at a single set of test scores, with no consideration of the education the student received in elementary or middle school or in the home. Typically, a high school can have only moderate effects on things like reading, yet many high schools get blamed for their low student reading scores, while others are praised unjustifiably for having the good luck to have well-prepared students entering their doors. Keeping track of the educational, ethnic, and socioeconomic background of students is especially important in a career magnet school, since its reading and math scores will be much more affected by its admission policy than by any pedagogical or curriculum choice it could make. The procedure we have used to compute a student's performance on a high school test net of what we would expect from their overall school scores is not very complicated; statisticians have been doing this sort of work for over half a century, and the capability is in the hands of every school district to produce these statistics. This is a politically sensitive issue; affluent school districts have little to gain and much to lose by measuring the "value added" by their high schools, while schools serving low income communities can only benefit, since they are being unfairly punished by the present system that gives them little chance to demonstrate any success.

      Providing accountability is only the first step. It is important to provide technical assistance and financing to programs so that they can attack the problem of maintaining high test scores while also running a strong job preparation mission. The schools in this study receive no special funding and no opportunity to experiment with new techniques to bridge the career-academic gap built into their mission. The amount of resources needed to do this may not be great. Career-preparatory high schools must realize that the majority of their graduates are going to go on to college. They are preparing students to hold jobs that will be used to pay college tuition. As more and more schools realize the implications of this, they will be more likely to want to provide their academic departments with the support and supervision that they need. It is also important that these schools be fairly compared to others so that any success they have in pushing their test scores up will be fairly recognized. Alternately, it may be that more high schools will decide that their only hope for providing both an academic and job preparatory education is to form partnerships with community colleges and lengthen the number of years of study. We cannot say with these data that the goal of meeting academic college requirements and preparing people for a specific skilled employment requires more than four years of high school. We can say that there is not much evidence that high schools operating without additional financial support are able to do this within the typical four-year time frame.

      The results of our study are complex and surprising. Experience suggests that academic career magnet programs will become increasingly common in the United States, and it is reassuring that this can be done without necessarily conflicting with the academic goals of the school. A properly designed academic career-focused program can prepare students for college more successfully than the typical comprehensive high school. In particular, our data show that a strong academic career focus can in some cases enhance the performance of students with low reading scores. In some cases, however, there are genuine conflicts between academic and career education; programs aimed at immediate job placement after high school may lower academic performance.

      We have not found a way to settle the argument about the proper balance between the academic and career aspects of high school education. Proponents of school-to-work will be heartened to learn that some components of academic-career integration have positive academic performance outcomes, and few have negative outcomes. Most academic career magnets do not show higher mathematics and reading performance, however. The mere fact that they do not lower performance will not be enough to satisfy some of the critics of school-to-work who believe that academic performance must be increased and that anything that does not help improve scores is a distraction from the main purpose of the school. Unfortunately, that issue is not magically resolved by these findings.

      We can also ask many questions about what it is about computers that lead to the enhancement of mathematics performance. The gains reported here are large--a gain on the Advanced Math Exam of one-third of a standard deviation (33 points on the math SAT), despite a 50% increase in the number of students taking the tests, is impressive. The improvement in attendance on the standardized examinations suggests that part of the impact of computers is motivational rather than strictly cognitive. What is their motivational value? Is it because they are linked to future occupations? Are students motivated by having control of a process, rather than being passive followers of instruction, or because of the physical nature of keyboarding and mouse-pushing? Are the computers a status symbol for students? If the computers have a more direct cognitive effect as well, is this because computers are used directly to reinforce mathematics lessons? Is there some sort of transference, perhaps because students are learning the step-by-step structure of computer programming and transferring that skill to mathematics problems? Are computers being used to handle numbers (as in accounting) or symbols (as in abstract programming) which make students more comfortable with either number-based mathematics or with algebra? Is computer work helping students lengthen their attention span? Is it because the physical movement of keyboarding reaches students whose intelligence is more kinesthetic? All these questions require more research, including more attention to how students use computers.

      There is seemingly strong evidence that programs that take students out into the workplace and prepare them for jobs immediately after graduation may have negative effects on student academic performance. Several other measures of employment focus, such as having vocational clubs, inviting guest speakers from industry, or using mentors, show neither positive nor negative cognitive effects in these data. There are strong noncognitive benefits attached to internship programs and mentoring, but no impact, positive or negative, on test scores. But one aspect of a "taking the school to the workplace program"--a commitment to placing students in employment after graduation--seems to lead to a de-emphasis upon academic performance. This may be the direct result of a competition for the student's time. In order to reach the level of performance required of an employer, the school must invest more time in training to meet those performance standards and, in some cases, this may lead to a sacrifice in academics.

References

Berryman, S. E., & Bailey, T. (1992). The double helix of education and the economy.New York: The Institute on Education and the Economy, Teachers College, Columbia University.

Crain, R. L. (1984). The quality of American high school graduates: What personnel officers say and do about it.Unpublished paper. Baltimore, MD: Center for Social Organization of Schools, Johns Hopkins University.

Glenn, N. D. (1994). Television watching, newspaper reading, and cohort differences in verbal ability. Sociology of Education, 67,216-230.

McNeil, Jr., R. B. (1995, January). Extracurricular activities and high school dropouts .Sociology of Education, 68,62-81.

Resnick, L. (1987, December). Learning in school and out. Educational Research,16,13-20.

Rosenbaum, J. E., & Nelson, K. A. (1994). The influence of perceived articulation on students' school effort.Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Rosenbaum, J. E., & Roy, K. (1996). Long-term effects of high school grades and job placement.Paper presented at the annual meeting of the American Sociological Association, New York, NY.

[3] Aggregation is also useful because, as Glenn (1994) notes, random fluctuations at the individual level will generally cancel one another out and not interfere with the correlation in the aggregated data.

[4] In Figure 3.1, the median score of all programs with computer usage above the mean (usage > 4) is -0.1; the median for all programs below the mean level of computer usage is -0.6; since these medians are not adjusted for the standard error of the estimates for individual programs, and the programs with the most extreme scores are no doubt those based on the fewest cases, this technique should give us a larger estimate of the effect of computer usage, and it does: .5 is greater than the .37 obtained in the individual level analysis in Table 3.1.

[5] This finding corresponds with that in Crain (1984) that unlike the situation for college graduates, employers of high school graduates ignore school grades. As a result, students not expecting to matriculate to college are not motivated to exert the effort required to achieve grades higher than those required for graduation. See Rosenbaum and Nelson (1994) and Rosenbaum and Roy (1996).

Crain, R. L., Allen, A., Thaler, R., Sullivan, D., Zellman, G., Little, J. W., and Quigley, D. D. (1999). The Effects of Academic Career Magnet Education on High Schools and Their Graduates (MDS-779). Berkeley: National Center for Research in Vocational Education, University of California.