Previous Next Title Page Contents Crain, R. L., Allen, A., Thaler, R., Sullivan, D., Zellman, G., Little, J. W., and Quigley, D. D. (1999). The Effects of Academic Career Magnet Education on High Schools and Their Graduates (MDS-779). Berkeley: National Center for Research in Vocational Education, University of California.

CHAPTER  1
Introduction

Robert L. Crain

      The United States high school is trapped in a dilemma. It must now teach all students because it is now widely assumed that every student should graduate from high school. Those who do not are called "dropouts," a term invented in the 1960s when not finishing high school became a failure to achieve the new norm. Furthermore, over 60% of high school graduates are enrolling in college immediately after high school. Now that high schools are no longer serving only the academically talented students, or at least the ones capable of obtaining the "gentleman's C" easily, the high school must find some way to motivate and teach students who are less well-prepared academically so that they will both want to and be able to graduate. But the high school is now chasing a moving target, as academic standards are being pushed upwards by political pressures of various kinds. Finally, there is pressure to increase equality of educational opportunity, and pressure from low-income parents who are convinced that their children must attend college if they are to survive economically in the modern world--which means that their children will not be going to vocational schools.

      This is a study of a group of high schools and programs within high schools in which students choose an academic career focus in eighth grade and leave their neighborhood for a regional career magnet school (or a regional program located in a distant comprehensive high school building). The careers the students choose usually require at least a community college education. The study suggests that however unlikely the notion of creating a school that is both career oriented and college oriented might seem, this is probably the type of school wanted both by the nation's leaders and by parents and students as well. Careers (if they can be made to work) may prove to be an ideal strategy for urban high schools. On the one hand, an academic career focus makes good sense for students who are unsure they will be able to graduate from college and have little access to occupational information either at home or in their community that will help them find quality employment. On the other hand, most of these students are not willing to trade the chance to attend college for even a good entry-level job. A high school that promises both college and a career allows them to make a choice without foreclosing their options.

      There is good evidence here that giving the college-preparatory high school an academic career focus creates a meaningful high school curriculum and provides a great deal of support to adolescents in this period of their development. Adolescent support is very important, and is an area where the traditional high school curriculum fails badly.

      These academic career magnets did not arise from some central planning process. They were the result of a gradual evolution. The district, with many poor and minority students, many immigrants, and much white flight, had many schools in crisis. In several cases, high schools were deemed to be such failures that they were closed and reopened with a new student body and staff. Other schools were closed because of declining enrollments due to demographic change or a loss of interest in vocational programs. Frequently, the school was located in an area where there were not enough students to support a school, and the school had to become a magnet. But vocational schools were not very popular, and there were already enough high school programs serving gifted students. By a mixture of default and foresight, local design teams of community residents, political leaders, and educators, who were given considerable power in the decisionmaking process, arrived at the idea of creating career magnet schools that were college preparatory without being restricted to the most highly talented students, attracting their students by having a particular focus--almost always a career. This academic career magnet strategy--we call it "career magnet" for short--became a successful recruiting device, and as the schools became popular and garnered more than their share of the city's best students, the principals of the comprehensive high schools fought back, creating their own "programs-within-a-school" in their high schools. By the late 1980s, a third of all the area's students were being educated in these schools and programs.

      In this area, the academically oriented career magnet may have been as much a bottom-up innovation as one imposed from the top down. The model has proven its political acceptability, its ability to garner support from parents and students, and its ability to be mass produced. The opportunity to evaluate program outcomes, to see if the model prepares a more skilled labor force, creates a more college-ready pool of high school graduates, or lowers the dropout rate, grew out of a happy accident: These schools, searching for the fairest possible way to admit students while being pressured to provide equality of educational opportunity, chose to fill one-half of those seats by creating a lottery system that made it easy for students to apply to the academic career magnet schools.


The Programs and Our Studies

      Career magnets recruit students by offering to prepare them for college and simultaneously provide them with an introduction to a particular career. In business, a program may teach accounting, financial management, word processing, and other business skills on the assumption that students will attend college, furthering their business education there. A typical program operates as a school-within-a-school, providing a number of classes separate from the rest of the school, typically teaching ten to twelve classes over a four-year period that are specifically focused on a career. Many have instructors with experience in that career. They take students on field trips to firms, and operate extracurricular clubs to learn more about careers. About half of them bring in lecturers from industry, use businesspeople as one-on-one mentors, or hold workshops on how to prepare a résumé. At the same time, they provide students the opportunity for a college preparatory education, and attempt to send a large number of students to community college. For many students, especially inner-city minority students, adult success may seem unattainable without college. They may live in a section of the city where few adults hold good jobs, and the only good jobs they know are held by people with college degrees. Attending a traditional vocational education program that focuses on employment rather than college would seem to be giving up too much, even for those who appear to have little chance of succeeding in college.

      The career magnet programs seem firmly committed to college preparation. For example, one career magnet program arranged its senior year schedule so that all students in the program spend one afternoon a week at a nearby college taking a course that will count toward their high school diploma and later as college credit. The program director describes this college course as popular with students, adding that it makes clear that career preparation is not incompatible with attending college.

      This report will focus on several research studies:


The Sites

      The 59 programs are located in 31 different high schools. Eight of the high schools are devoted entirely to career magnet programs, while the remaining 23 are comprehensive high schools that have a career magnet program within the school. For the survey of graduates and the life history interviews, we chose students who had applied to programs in four schools that were completely dedicated to academic career magnet programs, comparing lottery winners who had attended and graduated from the career magnets to students who had applied to the same four schools but had lost the lottery and had attended and graduated from comprehensive high school programs. We chose the four career magnet schools because they seemed to represent fully developed models. All four schools were at least ten years old and they collectively covered a wide range of career interests. We will refer to them as the "Health Careers magnet," the "Business magnet," the "Business Communications magnet," and the "Engineering magnet." In all cases, we were studying buildings that were entirely devoted to career magnet programs, even though the career magnet school within a comprehensive school is the more common model. We did so because we thought that the dedicated career magnet school building provided a clearer example that would be easier to interpret than the mixed model; we also thought that the school building devoted entirely to career magnet programs would be more typical of what other school districts in the United States would choose to implement. In each of the programs, we selected pairs of students who were matched on ethnicity, sex, achievement test scores, and neighborhood. One student in each pair had entered the lottery, attended the career magnet program, and graduated; the other lost the lottery, did not attend the career magnet program, and graduated from a comprehensive high school. Since we are not studying the students who dropped out, we are using a model that allows us to detect whatever differences occur when the two types of schools succeed in graduating a student.


The Schools

      "Health Careers" is a career magnet with only two programs: A small program, medical science, which focuses on theoretical medicine, seems primarily intended for college-bound students who are likely to enter nursing or pre-med programs in four-year colleges. The rest of the school is used by a large health careers program that includes eight different components, each preparing students for qualification in a particular area: practical nursing, nursing assistants, dental assistants, dental laboratory work, medical laboratory work, medical accounting, medical office work, and medical secretarial work. The wide variety of options within the program allows the school to serve students who vary greatly in their ability to work in front-line hospital service. Those who cannot tolerate blood or who cannot function safely in a clinical setting are provided a variety of office alternatives. At the same time, the academically strongest nursing students may move into the medical science program.

      The "Business magnet" high school contains seven business-oriented programs. The most prestigious is its program in securities and finance, but equally important are programs in accounting, business, computer science, business law, marketing, and secretarial science, an information systems program, which prepares students to work in a variety of word-processing positions. The presence of seven related programs gives the school the opportunity to reduce its dropout rate by moving students among the programs if they lose interest or are unable to do the work in one area.

      The "Business Communications magnet" is in many ways a similar school, but because it lacks the high-prestige finance program, it probably does not attract as many students interested in attending Ivy League colleges. Its three programs are (1) computer programming/accounting, (2) marketing, and (3) secretarial studies and word processing careers. Its advanced accounting students can do coursework at a partner college.

      The "Engineering magnet" has four programs. Two of its programs are in aerospace technology and computer science. In contrast, its other two programs, one in law and the other in television and other communications, provide a liberal arts complement. The student body is economically heterogeneous but entirely African American. Two of the programs have partnerships with local colleges: (1) The aerospace technology program is designed for students going on to engineering school and provides considerable work in engineering design. A small coterie of students are provided the opportunity to learn to fly. The program also offers opportunities to learn about the operation of electronic equipment and the maintenance of airport facilities; and (2) the communications program's main focus is to provide students with the opportunity to operate a television production facility in the school. However, it does not have good connections to the local television industry since its students and graduates must compete against college graduates even for unpaid internships. The other two programs do not have college partnerships in place. Computer science, which is primarily mathematical in its orientation, provides more computer theory than practice. The law is a popular subject with high school students. The program has a variety of internships, and it holds a mock trial competition each year. As with many of the programs, the academic career focus is broad, including preparation for police work and other aspects of law enforcement as well as preparation for college pre-law programs.

      All the career magnet schools must cope with the extremely wide range of students that they have, and most do so by moving students between programs. Sometimes a single program will be stratified, with a small group of students in the most advanced courses and others in an easier curriculum.

      Most career magnets receive no additional funds from the school board (a few have federal or foundation grants) and must reallocate funds within their regular budget to pay for any special equipment the programs require. The only exception are "redesign" funds, which become available to a school after it is deemed to be such a failure that it is closed and reopened with a new name and a new staff. These schools receive a supplemental appropriation for the first five years of their new life. Some total academic career magnet schools began their life as "redesigned" schools, making them the only academic career magnet programs that received extra funding.

      These career magnets provide a good opportunity to isolate particular elements of school-to-work programs, since some of the programs emphasize computers more than others, stress more visits to firms, or use more teachers with backgrounds in industry. The district's management system for these academic career magnets is highly decentralized. As a result, each program is free to create its own particular theme and to decide for itself many of the details of its curriculum and structure.

      While our study is of schools that are focused by their definition, the fact that the focus is specifically career-oriented limits the extent to which this study can be considered an evaluation of schools with focus. Nevertheless, it will contribute to our understanding of those schools as well as to the discussion of strategies for school-to-work transition. Its data will also be useful to the policymakers concerned with "school to work" and especially with "school to work for the college bound" (Bailey & Merritt, 1997) and "academies" (Kemple & Rock, 1996; Stern, Raby, & Dayton, 1992). This study also contributes to the research on choice, since these schools were constructed as choice schools--being magnets first, with the choice of theme coming second. Indeed, some of the programs created in this effort have no academic career focus at all.

      Finally, these programs in some ways resemble charter schools. Although they do not have charters and are not officially deregulated, all the high schools in this area are surprisingly autonomous. Decisions about what kind of program to operate, what changes in program should be made, and what new programs should be added are made with very little control from any school board or higher government administration. Nowhere in local or state government is there any educational administrator specifically responsible for career magnet programs, and, indeed, the official descriptions of the schools in this area do not even recognize the career magnets as a distinct type of high school program. Further, records do not identify whether students are in a particular program but only the building that they are in, so that a student in a career magnet program in a comprehensive high school is not identifiable as such statistically. This means that these schools are quite free to do as they wish, with little opportunity for the school system to regulate them through controls or even evaluate them. The schools are bound by the standard regulations governing personnel, which sometimes cause serious problems when schools need to recruit faculty who have specialization in their particular careers. But the looseness of these high schools, carried almost to an extreme, may mean that these schools provide interesting lessons for persons concerned with how charter schools may work out.


The Selection Process

      When the district's first academic career magnets were designed, it was intended that they be like most magnet schools in America--selective. However, the school board and administration has probably been more sensitive to issues of race and class segregation than most school districts in the country. When critics made the school board aware of the conflict between the magnet school's goal of being selective and the social goal of furthering racial and economic integration, a compromise strategy was gradually worked out that noticeably reduced the segregative nature of the selection process. First, in what appears to have been an effort to encourage students to apply to magnet programs, every middle school student in the area was required to fill out an application for high school (even if they planned only to attend their neighborhood high school), using a form that made applying to a magnet school as easy as possible. Secondly, each academic career magnet program could admit only one-sixth of its students from those with above-grade-level reading scores in the city's student body and another one-sixth from those reading below grade level (the other two-thirds coming from those within one standard deviation of the area mean). Finally, they required that half of the students in each of those three reading groups be admitted by lottery. That was a political compromise, since the principals of the career magnet high schools wanted to select all their students and the critics wanted them all randomly assigned; 50% was an obvious compromise point.

      If students wished to be considered for career magnet programs, they listed up to eight in order of priority. If they were also interested in programs at the four most highly selective schools in the area, they checked a separate set of boxes on the questionnaire; this arrangement meant they did not risk wasting their first priority choice by betting on a long shot. Finally, if they wanted to attend their neighborhood comprehensive high school, they could simply check a box on their application. The application process was spread over the entire fall semester of the eighth grade, giving students a chance to change their minds and to discuss choices with their parents.

      In late January, a subcontractor selected enough students to fill half the seats in each career magnet program. They did so by assigning random numbers to each student's choices, but preceded each random number with a number that represented that student's priority of choice. Thus, each student's first choice began with the number one, their second choice with the number two, and so on. The students with the lowest random numbers choosing each program were automatically offered admission. A waiting list was also created to replace decliners. Separate lotteries were performed for students with high, average, and low reading test scores. High was defined as one standard deviation above the mean, low was defined as one standard deviation or more below the mean or having no test data. Of the students randomly selected for each program, 16% were from the high- and another 16% from the low-reading groups, with the remaining 68% from the average reading group. Students whose test scores fell in the top 2% of the test distribution were exempt from the lottery and given their first choice assignment whenever possible.

      The files of students who had not been selected by lottery, including those who had been wait listed for the program, were then sent to the school, which selected enough students to fill one-half of the seats in the entering class from the remaining students. Students admitted to a program on the basis of their test scores being in the top 2% were counted as part of the top 16% of those selected by the school.

      The studies reported on here are based on a cohort of 9,174 students who applied to 59 different academic career magnet programs. (Students are also admitted into these programs in the tenth grade if they attend a seventh- through ninth-grade junior high, but we did not study these students.) Of these students, 2,373, or 26%, were admitted by lottery to their first choice academic career magnet program; of these, 63% entered ninth grade in that program. Of those who lost the lottery, 18% were school-selected and attended their first choice academic career magnet program. The remaining lottery losers were either admitted to one of the elite public high schools, a career magnet program that was not their first choice, entered ninth grade in their neighborhood comprehensive high school, or withdrew from the public school system (see Table 1.1).

      Of the students applying to these programs, 61% were female and 39% male. This imbalance may be because this sample of career magnets often focuses on careers in health and business, which are attractive career areas for females. It may also be that females are
more future-oriented at age 14, or that boys want to attend their neighborhood high school while girls want to escape from their neighborhood at that age.

      Student ethnicity is coded by the school board into five categories. For the cohort reported on here, the ethnic breakdown is as follows: 47% African American, 27% Hispanic, 8% white, 5% Asian American, and 1/2% Native American. Information concerning ethnicity is voluntary, and 986 students (12.5%) were not ethnically classified.

      We used the school district's method of assigning students by lottery to career magnet schools as the basis for our study. It is not immediately obvious that the lottery admission used in the school district was indeed a randomized experiment, and, in fact, it differs in certain ways from a traditional experimental design. But it does meet the two necessary conditions: (1) that subjects be randomly assigned to different treatments and (2) that outcome measures be taken after they have received the treatment. Lottery assignment to oversubscribed programs meets the first condition because it guarantees that some students will be randomly admitted to a particular program while other students who also applied to that program are randomly rejected. In all of our research we were studying only students who were lottery admitted to career magnet schools and graduated from them, comparing them to lottery-losing applicants to the same schools who had graduated from comprehensive high schools.

      We refer to the study as an experiment-based study rather than a classical laboratory experiment because we could not include every student who participated in the lottery. We could not study the effects of the programs on the students who left the study, attending private school or leaving the school district after having participated in the lottery. We could not study the postgraduate performance of students who did not graduate from the schools. In addition, when we did use the lottery as an experiment, we had to allow for the fact that some students were "misassigned," some lottery losing students attended the career magnet program despite having lost the lottery, and some lottery winners chose to attend comprehensive high schools. These omissions mean that our study should be thought of as based on an experimental model but not a perfect experiment. We have been as rigorous as possible in retaining the power of the original random assignment. Since in its initial step the lottery is exactly the same as the first step of a randomized experiment, we have stayed as close as possible to that original population.

      We examined the data for the first two years of the lottery (students admitted in 1987 and 1988), and students entering the fourth year in Fall 1990. The 1987 data showed deviation from randomness: lottery winners and lottery losers differed in their middle school performance to a significant degree. These differences were small but with an overall sample size of over 47,000, small differences are nevertheless statistically significant. The 1988 lottery selection showed a much smaller bias, although still statistically significant, and the 1990 lottery process shows no statistically different results at all. We spent the first year and to some extent the second year debugging the system until nonrandomness was eliminated. In our study, we used the data for the 1988 entering class of ninth graders.

      For each of the three reading groups and for every program (there were 136 programs in 1988) we identified the number of lottery winners and lottery losers and counted the number of students who actually entered the program after being admitted randomly. Since the lottery selection used the lowest random numbers, every student who applied to a particular program as his or her first choice had priority over students who applied to that same program as their second or higher choice. We then eliminated every program with fewer than nine students admitted to the program randomly and fewer than nine rejected or placed on the waiting list. We also eliminated programs where fewer than 60% of those randomly admitted actually attended the school, and those where more than 40% of the lottery rejected were selected by the school for admission after being rejected by the lottery. We thought the data from these programs would be meaningless because of so many assignment errors. In a few cases, we found that the entire set of first priority students were admitted because there were fewer students choosing the school as their first priority than there were seats available for lottery admission; in these cases, we looked to the students who had selected the school as their second priority choice to see if an experiment could be constructed using only the second priority choice students. We assumed that students who selected a school as their first priority are different from those who select a school as their second priority, so that simply comparing all lottery winners to all lottery losers would bias our sample, with the lottery winners being more likely to be first choice and the lottery losers to be later choices. For this reason, we made all our experiments within a single priority choice.

      When the selection of all experiments was completed we found that we had identified 112 experiments in 59 different programs involving 9,174 students. Some of the 59 programs were so large that we had valid experiments for all three reading levels, but more often a program provided valid data for experiments at only one or two reading levels.

      The first studies that we conducted were analyses of the effects of being randomly admitted to a career magnet program on the academic performance of students--their official school records of test scores, absenteeism, graduation, and dropping out. We also identified those characteristics of career magnet programs that had the largest impact on the students--either by increasing or decreasing the performance of the career magnet students compared to the lottery losers who had applied to the same program. For example, we found that when students applied to a program that provided students with more opportunities for hands-on computer work, lottery winners had higher math test scores than lottery losers, implying that the computer time was the factor that improved students' scores. We also used this sample to study graduation rates.


Methodology for the Survey and Life History Studies

      In the studies reported here, we turn from the general academic data on students to personal interview data. We conducted a set of interviews with a subsample of 110 graduates of career magnets and regular comprehensive high schools. In addition, 13 matched pairs of this subsample were asked back to give us life histories and to let us observe them as they worked together on a team work project.

      We concentrated on four schools, each made up entirely of career magnet programs. We did this because we expected schools entirely dedicated to career magnet programs would have more administrative support because they did not have to compete with a large comprehensive program, and, thus, would be able to focus on making their programs effective. We wanted to compare students who were admitted by lottery to the four schools, and who had subsequently graduated from them, to students who had lost the lottery, attended a comprehensive high school, and had graduated from there.

      We drew a random sample of the lottery winners and losers and deleted everyone who had not graduated from the high school within five years. To make the two groups as similar as possible, we selected graduates matched on the program they applied to, their home neighborhood, their test scores, ethnicity, and gender.

      Locating prospective interview respondents and gaining their cooperation was complicated by regulations covering issues of confidentiality, which required that the school district obtain each student's permission before releasing the student's name and address. However, the Board of Education's research office was too overworked to accept a subcontract from us to do this. As a result, we employed a guidance counselor from each school to make the initial contact and obtain permission for us to follow-up with potential respondents. This turned out to be a difficult process and by the time school ended for the academic year, only one-third of the prospective respondents had agreed. We had originally selected 483 graduates, but wound up interviewing only 110. This does not represent a low response rate (nearly everyone with whom we made contact agreed to the interview, for which they were well paid), but a low rate of success in contacting them initially. There were a number of reasons for this, the most common being that the school address for a student was wrong or because the guidance counselor was unable to make the repeated attempts that would have been necessary to contact them.

      Of the 110 graduates in this subsample, fifty-one had "won" the lottery and attended, and graduated from, their first choice career magnet. The other fifty-nine had "lost" the lottery and graduated from a comprehensive high school. All were between the ages of 19-22 years (mean age was 19.8) at the time they were interviewed. Respondents in this subsample identified themselves as follows: 46% African Americans, 3% Asian Americans, 37% Caribbean Americans, 12% Latino Americans, and 4% multiethnic. (Because there are few whites in the district schools and we thought they would be quite different from minorities in their school and post-high school experiences, we decided to select only minorities.)


The Survey

      The interview combined the qualitative richness of open-ended questions with the quantitative analysis possibilities of closed-ended questions. This format was chosen because we wanted standardized data, but discovered from our initial pilot interviews that much of the data we were seeking was too complex or threatening to gather in a closed-ended manner. The instrument was designed to capture differences in how graduates experienced their high school classes, peers, teachers, counselors, and the school as a whole; their employment history; career development; ethnic identity; and lifestyle choices such as alcohol consumption, drug use, and contraceptive precautions. Following the interview, respondents completed internal locus of control and Rosenberg's (1965) self-esteem instruments. This interview ranged from one-and-a-half to two-and-a-half hours in length. All interviews were tape-recorded for later assessment.

      Four African-American graduate students (3 female and 1 male) interviewed the respondents over a nine-month period after being trained in listening and interview techniques. Anna Allen interviewed the three Asian-American respondents and the one white respondent who fell into the sample by accident (her ethnicity had not been known when she was sampled). Most of the interviews were done by an interviewer of the same sex. Respondents were paid, including a bonus for travel time.

      All interviews were tape-recorded. In general, respondents were encouraged to talk as much as possible, and no effort was made to limit the length of these interviews. Over 95% of the questions were open-ended, and responses were recorded verbatim. Post-interview coding led to some questions having as many as 50 response categories.

      The instruments took the respondent through the years from eighth grade to the present, asking their recollection of why they chose their high school program, what their attitudes about college were at that time, what their work experience and career goals had been, what their socio-emotional strengths and problems had been, and the amount of support they received from their school staff. For the high school period, they were asked about their peer groups, the kinds of friends they had, the amount of support they got from the work that they did, and the connection of that work to school. A large number of questions dealt with the connections between school and work and the degree to which classes integrated school and work. There were also questions about community service.

      Students evaluated the amount of support they received from teachers, and evaluated themselves in terms of their greatest accomplishments and difficulties, their level of confidence, and any problems they had in school. They were asked whether they were under peer pressure to perform poorly in school, how much counseling they received from staff about college and career, and what advice they received from significant others in their family and social group about life choices. They were asked their opinion of the teachers they had and why they did or did not like them.

      They were asked why they had chosen the career that they had in mind; whether the high school gave them opportunities to have a mentor, to job-shadow, or to hear speakers about work; and generally how they felt about the amount of information that was available to help them make career decisions. They were asked to describe their present job--its strengths, weaknesses, and general characteristics; how many times they had changed jobs and why; what their future job aspirations were; and their evaluation of their work skills--their strengths and weaknesses and their performance compared to fellow workers. They were asked to describe the skills needed for their particular job, so that their responses could be related to SCANS criteria.

      Next, they were asked about their college plans in high school, and whether their parents were willing to provide financial support. In terms of college, they were asked what their grades were, the amount of homework they did, what difficulties or problems they had, and what their strengths and weaknesses as a college student were.

      In the last section, they were asked a number of personal questions about what they like to do; their sense of happiness; their sexual behavior, use of contraception, and whether they had conceived a child; their alcohol and drug use; and their history of fights, arrests, and victimization. They were also asked about their family structure, their parents, older and younger siblings, the amount of contact they presently have with their parents, the amount of stress in the family, feelings they had about the work their parents and siblings did; the family's problems, activities, and the role of punishment and violence; and the family's experience with welfare. They were asked about their marital status, their use of child care if they had children, their religious behavior and the support they received from their religious organization, and their political views and racial attitudes.

      In general, questions probed much more toward the affective side of their opinions--questions like "What do you love to do?" were asked as well as "What do you like to do?" A number of questions gave them an opportunity to talk about emotional problems and about support from various others, as well as lack of support. Given the context set up by the instrument, we are optimistic that their answers about fighting; their arrest records; their victimization; and their use of cigarettes, alcohol, and drugs are reported more accurately than in most surveys.

      There was no significant difference in the ethnic make-up of the comparison groups in the survey and life history interviews (although there are more students in the control group whose families do not use English at home). The original experimental plan was to create pairs matched on ethnicity, gender, same choice of career magnet program, eighth grade reading scores, and the junior high school they came from. This procedure provided us with too few possible pairs, so the junior high school of origin criteria was broadened to include other junior high schools in the same general neighborhood. In the final sample, lottery winners and lottery losers were similar in most important respects. Demographic analysis reveals no significant differences by sex, ethnicity, eighth grade reading scores, or parental education, but one test (seventh grade math) does favor the career magnet graduates. Measures of reported socioeconomic status (SES) show no appreciable difference between the two groups. All had entered high school in the Fall of 1988 and all had graduated by 1993. None had been in special education.

      Table 1.2 shows no significant bias between the two groups, but does show that both groups are heavily female. This is partly because our sample of good experiments included more females than males, and partly because these four career magnets emphasize careers that tend to attract female students. However, the fact that the sample is two-thirds female does not bias the comparisons of lottery winners to lottery losers since females are overrepresented in both groups.

      After interviewing them, we discovered that 11 of the 59 lottery losers had been admitted to career magnet programs within their comprehensive high schools. Table 1.2 shows a significant bias. Graduates from the comprehensive programs had lower seventh grade math standardized test scores than did the lottery winners.


Life History Interviews with 26 Graduates

      From the 110 graduates who participated in the two-hour "short" interview, we selected a subsample of matched pairs for a semistructured life history interview. We were able to successfully interview only 13 matched pairs; six others who were possible matches cited various reasons for not participating. There are no discernible differences between those not re-interviewed and those who were re-interviewed. This subsample of 26 graduates was composed of five pairs of males and eight pairs of females; four non-matching respondents were also interviewed, bringing the sample to 30.

      The life history/career development interviews were approximately four hours long and, when possible, conducted over two different days. The first two hours were devoted to a life history and the second two to tracing career development and the respondent's activity after high school. The interview guide began with general background information about the respondent and his or her family prior to entering school. It included specific questions about the family and educational experiences for each chronological year of school from kindergarten until the time of the interview. This procedure provides a complex description of how school and home interacted for each respondent to affect their career development and what meaning each graduate retrospectively attaches to these experiences. Asking each respondent specific questions about each year allowed us to look for patterns across cases.

      Each interview typically produced a transcript of slightly over one hundred pages. Transcripts were coded using an ethnographic program, "Atlas/ti," recommended to us by Matthew Miles. Judging from the number of stories of childhood trauma and descriptions of misbehavior, this interviewing procedure seemed successful in getting candid histories.

      Whenever possible the interviewers who had interviewed the respondent for the semistructured interview did the life history interview in order to take advantage of the rapport established during the first interview. Each case was coded and analyzed by the graduate student who did the interview. Cases and strategies were discussed in frequent meetings with the authors.

      Since the 30 respondents provided both the interview and the life history, it was possible to mix the quantitative analysis of the interviews with the ethnographic analysis of the life histories.


Bias

      While this is not a perfect randomized experiment, it comes much closer than traditional longitudinal studies. Students had been randomly assigned to one of two groups, either winner or loser, through the administration of a lottery. They had chosen the same programs; were in the same reading ability groups; and, in the case of the survey, shared the same talent and tenacity necessary to graduate from a high school. We think the most important attribute of this study is that it evaluates schools that are in the real world, to which students were assigned randomly. The schools are not pilots or experiments; they have been mass-produced and have passed the test of feasibility. The use of lottery admission gave us the opportunity to base our analyses on an initial random assignment of students. While we could not achieve the rigor of a laboratory experiment such as might be done in medicine, we think the experiment-based statistical analysis here, using the lottery admission to these high school programs, provides us a much stronger analysis than has been done before.


References

Bailey, T., & Merritt, D. (1997). School-to-work for the college bound (MDS-799). Berkeley: National Center for Research in Vocational Education, University of California, Berkeley.

Kemple, J. J., & Rock, J. (1996). First report on the career academies demonstration and evaluation. New York: Manpower Development Research Corporation.

Rosenberg, M. (1965). Society and the adolescent self-image.Princeton, NJ: Princeton University Press.

Stern, D., Raby, M., & Dayton, C. (1992). Career academies: Partnerships for reconstructing American high schools.San Francisco: Jossey-Bass.

Previous Next Title Page Contents Crain, R. L., Allen, A., Thaler, R., Sullivan, D., Zellman, G., Little, J. W., and Quigley, D. D. (1999). The Effects of Academic Career Magnet Education on High Schools and Their Graduates (MDS-779). Berkeley: National Center for Research in Vocational Education, University of California.
NCRVE Home | Site Search | Product Search