What has become apparent from reviewing vast literature is that further work is needed to sort out the complicated and often contradictory findings regarding the economic returns to vocational education. Many of the studies reviewed reveal that it is the interaction of a variety of factors that predicts economic success. Until researchers can isolate those effects, or better understand the interaction between key factors, conclusions about the effects of vocational education will remain tentative. (p. 405)This conclusion should not be permitted to become an excuse for inaction because we do not know enough yet (see Grubb, 1995; Kane & Rouse, 1995a, 1995b). Responsible authorities must act now, even if the validity of a relationship between a particular education event and earnings has not been established. Here, validity refers to evidence that a measure--such as earnings--actually represents a vocational education outcome. Some confuse this concept with a measure's reliability, which represents the strength and consistency of evidence that a measure is accurately recording what the investigator wants to record. For instance, it might be concluded that a State Employment Security Agency's database is a reliable source to document a former student's earnings level, while believing that this is not a valid measure of a vocational education outcome. This view might be expressed if one thinks that these measures also reflect the influence of other factors, such as ability, motivation, previous employment experience, other education exposure, and work-site circumstances, which make it difficult, and perhaps impossible as a practical matter, to isolate vocational education's independent influence (see National Institute of Education, 1981, p. VII-22; and Wirt, Muraskin, Goodwin, & Meyer, 1989, p. 122, for a historical perspective).
Figure 1 provides a visual representation of a metaphor that describes the interplay among these three basic concepts. The importance of this interdependence must be understood to appreciate the relevance of the outcome measurement lessons that appear in the next two sections.
The metaphor used here is optics. Clear vision is a function of complex forces that remain a mystery to most of us. Similarly, a former student's subsequent employment status is determined by their own qualifications and actions, the qualifications and actions of others, employer requirements, and employment opportunity. Aspects of this interplay remain as mysterious to many people as the phenomenon of sight.
Pursuing the optics metaphor, think of a former student's decision to seek employment as a pulse of light, and an employer's decision to hire a new employee as a second pulse of light. Figure 1 shows each of these beams of light passing through two screens before arriving at a target. The human eye passes light through pupil, lens, vitreous fluid, and retina before it is translated through the optic nerve to the brain. Each component has a distinct function that contributes to the quality of perception.
A person's decision to look for a job is screened through their own unique bundle of qualifications, and those of all others who might be thought of as competing candidates, before arriving at the employment opportunity destination. An employer's decision to hire a new employee is screened through their own specification of requirements, and the requirements of other employers who are competing for available candidates, before arriving at the hiring opportunity point. The roles of each of these screens and the employment opportunity set for investigating vocational education outcomes must be understood before proceeding to the examples that appear in the last two sections of this guide.
Historically, a common candidate qualification criterion has been evidence of having reached an educational plateau such as high school graduation or receipt of a postsecondary certificate or degree. However, as the pool of those who have reached each of these plateaus has grown, the relative importance of this qualification criterion has fallen. The value of an attribute as a discriminator among candidates falls as the fraction of those who offer the attribute rises. At the extreme, when all of the candidates offer an attribute, it cannot serve as a basis for selection. And, of course, if no candidate offers the attribute, it cannot serve as a discriminator. This is not the same as saying that an attribute cannot be a requirement when all candidates exhibit it. Achievement of the goal of universal literacy by the Year 2000 would not mean that literacy would then become irrelevant.
A less common screening criterion, but one that is being promoted as a prime candidate for widespread use, is certified competency. The National Skills Standard Board established pursuant to the Goals 2000: Educate America Act is attempting to hasten the obsolescence of employer reliance on evidence of program completion alone as a candidate qualification criterion. Vocal advocacy will be heard for the substitution of evidence of actual skill competencies that are consistent with established industry standards or requirements. Another pertinent qualification criterion is an employer's preference for relying upon referrals from current employees, based on a belief that incumbent employees know the qualifications that are appropriate and that they have a selfish interest in screening out poorly qualified candidates whose failure on the job would reflect on their own judgment.
Whether consideration of a particular candidate qualification criterion is common or unusual may affect how wage record data will be interpreted as evidence of a vocational education outcome. At higher levels of geographic aggregation, the relative importance of an unusual screening criterion will recede. For example, a particular classroom teacher may have earned a local reputation for attracting highly motivated and competent students who are aggressively recruited by local employers. This reputational advantage will be reflected in the employment and earnings records of this teacher's former students. However, at higher levels of aggregation, such as the district-wide performance of peers who were exposed to a similar curriculum, this advantage will be offset by average and below average achievements.
It is important to consider how employer use of illegal candidate qualification criteria might affect the interpretation of wage record data as evidence of a vocational education outcome. A school administrator's awareness of pervasive discrimination practices in a particular employment sector might be expected to result in either, or both, of the following undesirable policies.
These examples indicate how employment and earnings data might be used to help, or to hurt, a vocational education program. Fear of the unknown, and concern about a loss of control over performance measurement, has led some vocational educators to oppose the acquisition and use of this data.
The distinction between persistent and periodic candidate screening criteria promotes a similar emotion of uncertainty, or fear of misuse of data. Qualifications that are sufficient for successful candidacy at one time, or in one place, may not suffice at a different time, or in another location. Economic conditions vary across locations at any point in time, and over time within a particular location. When these differences, or changes, are large they strain a vocational educator's ability to calibrate their offerings to current standards.
Wage record data offer an unprecedented opportunity to monitor the relationship between candidate qualifications and employer requirements, but this relationship only has meaning in the context of the third component of Figure 1--the employment opportunity set. An alignment of qualification and requirement is sterile in the absence of opportunity. This can be illustrated by returning to the optics metaphor. The pupil, lens, and vitreous fluid may be normal in both right and left eyes, so light is properly focused on the retina; however, if either retina (i.e., the metaphorical target in Figure 1) is damaged, the brain's perception will be distorted or destroyed. Similarly, candidate qualification bundles may be accurately recorded, and employer requirements may be defined with equal clarity, but in the absence of employment opportunity each is meaningless.
No one argues that decisions about the level and type of vocational education investment should ignore current and projected labor market conditions. Discontent arises from what are considered to be unreasonably short time horizons for adapting curricula and enrollment flows to new economic circumstances. No one advocates an exact alignment of skill acquisition and use. Instead, displeasure is expressed about recurring patterns of nonuse of skills that are expensive to develop.
The candidate qualification, employer requirement, and employment opportunity components of Figure 1 can now be interpreted using the optics metaphor. A person's decision to look for or keep a job, or to seek a promotion, is based on an assessment of their current bundle of qualifications and the perceived qualification bundles of competing candidates, both interpreted in the context of speculation about the employment opportunity set. A candidate's decision to act is characterized as a pulse of light with four descriptors:
This part of Figure 1 also conveys a strong message that the expected impact of one spell of exposure to vocational education must be interpreted in the overall context of a student's own previous achievements, the current qualification bundles offered by competing candidates, and the current reservoir of employment opportunities. Each reader is urged to consider how a particular vocational education event might be expected to affect the timing, intensity, trajectory, and sustainability of a student's competitiveness. Four scenarios illustrate the range of conclusions about vocational education's impact that might be reached: (1) A high school student who has completed an integrated multiyear program of vocational courses without accompanying work experience; (2) a successful completer of a three- or four-year Tech Prep curriculum with related employment experience; (3) an employed adult who has completed three seemingly unrelated community college courses to qualify for promotion within the company; and (4) an employed, or unemployed, adult who has returned to a community college to complete one or more modules of vocational courses to prepare for a career change. Use Figure 1 to decide how vocational education might affect each student's pulse of light (i.e., candidacy).
The lower half of Figure 1 represents the demand elements of what has been labeled "a dynamic context of employment opportunity." The optics metaphor should be familiar enough by now that this part of the story can be told more quickly.
An employer's decision to attempt to hire a new employee, or to retain or promote an incumbent, begins with a specification of the requirements that candidates will be expected to meet or surpass. This specification takes into account both the threshold requirements of the position itself and what is known about the requirements of other employers who are expected to compete for the same population of candidates. These two screens are identical in function to those in the upper half of Figure 1; they affect whether, when, with what intensity, and for what duration the employer's action appears on the employment opportunity target. Here, ricochets represent requirements that are out of sync with the actual demands of the job, or with the requirement bundles that are used by competing enterprises. An employer's transmission of a pulse of light that is ill-timed, or that does not persist, is less likely to find qualified candidates on the employment opportunity target.
Think of the employment opportunity set as a stage floor, with a student's candidacy for employment and an employer's announcement of a job opening as spotlights that have the capability to sweep this stage. A match occurs only if these two spotlights overlap on this stage above a minimum level of illumination. Timing, intensity, aim, and persistence each contribute to the likelihood that a match will occur. The stage (i.e., the employment opportunity set) may be large or small; there may be many job opportunities, or only a few. The sweep of either of the spotlights may be limited, which means that the likelihood of overlap is reduced or even eliminated. This will happen if the qualifications of available candidates are inconsistent with the requirements of available jobs. Persistence on either side becomes important if time is required for either party to modify their bundle of qualifications/requirements.
Visualize a student's emission of a pulse of light being reflected back by the pool of available candidate qualification bundles. This describes a case in which a student's decision to seek work with a current bundle of qualifications is rejected by the availability of better qualified candidates, or by candidates with similar qualifications who are willing to accept a less costly compensation package. The student must then decide whether to try again at another time or in a different local labor market, to upgrade their own bundle of qualifications, or to modify their compensation requirements. Similarly, when an employer's announcement of a job opening is reflected it means that their bundle of requirements is out of sync with those broadcast by competing enterprises, so they have to decide whether to modify their expectations, seek candidates elsewhere, withdraw the opening, or sweeten the compensation offered.
The fundamental theme of this optics metaphor is that each vocational education event has one impact on one qualification factor for one candidate. The remaining two sections in this guide describe how a state employment security agency's administrative records can be used to document elements of this impact.