EMPLOYMENT AND EARNINGS CONCEPTS,
MEASURES, AND THEIR USE

• Employment Concepts
• Why Recording a Single Snapshot of Employment Status
  Should be Avoided
• Employment Measures
• Earnings Concepts
• Earnings Measures
• Mobility and Its Consequences
• Measurement and Interpretation of Mobility

Employment Concepts

Evidence is presented later in this section that challenges continued reliance on placement as a single, or even primary, measure of employment outcome. There are at least three good reasons why caution should be exercised in the use of a placement measure.

1. Some students maintain an employer affiliation after leaving school that began while they were still enrolled in school, or even before they were enrolled. This means that the basic concept of placement is irrelevant in such circumstances. This pattern is particularly frequent among former community college students. The recorded employment status is not an outcome at all, unless the observed continuity of employment is thought to have been contingent upon the employee's participation in the reference vocational education activity. Also, in such cases, the phrase transition from school to work mistakenly identifies simultaneous activities as sequential events.
   
   
2. Employment status at any particular time is a joint outcome of many forces, only one of which is a vocational education activity in which a former student may have participated. There are many reasons why encouragement of documented employment at a particular time may have undesirable consequences. For instance, awareness that employment status during a particular interval following graduation will be used as a performance measure provides an explicit incentive for authorities to act to assure that as many of the former students as possible will be counted as employed at this time.
   
   Former students are likely to be encouraged to accept inappropriate jobs just so they can be recorded as employed at the designated time. Such distortions can then have long-term consequences if this job limits a former student's subsequent opportunities to advance, if it promotes an attitude of disappointment with the reward realized from investing time and emotion in the pursuit of additional education, or if it establishes a pattern of lateral job-hopping without advancement.
   
   
3. Reliance on a single snapshot of employment status, such as the January-March quarter following the year of graduation, can introduce seasonal, and even cyclical, distortions that reflect differences in local labor market conditions that are likely to be unrelated to vocational education's potential long-run contribution to the well-being of former students, the local community, and the Nation.

1. High School Students--At the outset, employment during the last months of school enrollment should be documented. This provides a benchmark against which subsequent employment can be gauged. Continuity of employment through the bridge period of leaving school can then be identified, which makes it possible to investigate the interplay of concurrent employment and enrollment in classes. Patterns which reveal how one type of affiliation while in school is often associated with a different, but predictable, affiliation after leaving school can also be identified.
   
   Repeated measurement of employment status following a former student's school leaving can support a capability to then match employment records with postsecondary education records. Caution must be exercised in doing so. Unobserved enrollment in out-of-state postsecondary institutions can affect an analyst's interpretation of observed education/employment pairings and earnings relationships. Evidence is presented in the next section that demonstrates how combinations of employment and continuing education can be documented. This is especially important to avoid mistaken attribution of observed employment status and earnings levels as outcomes of high school vocational education alone.
   
   
2. Community College Students--Everything that applies to high school students is pertinent here, too; however, knowledge of pre-enrollment employment status is of interest as well. Many community college students have established records of employment that must be considered as a joint-input when conclusions are reached about the relationship of education and subsequent employment.
   
   Documentation of previous postsecondary education is also important. Careless reliance on an assumption that the award of a community college degree, certificate, or other type of recognition of achievement is a student's highest level of accomplishment is increasingly unfortunate, as many recipients of baccalaureate and graduate degrees have returned to community colleges to update or extend their skills.
   
   
3. Baccalaureate/Post-Graduate Students--Again, overlays of pre-enrollment, concurrent, and post-schooling employment with the full series of educational affiliations would be best suited for a serious attempt to isolate the effect of any one component of this investment series on a former student's subsequent employment history.

Why Recording a Single Snapshot of Employment Status
Should be Avoided

Figure 2 illustrates how this traditional approach can be refined to more accurately reveal the relationship between a former student's participation in a vocational education activity and an observed employment status. Beginning in the upper left corner of Figure 2, it is important to identify all members of the reference population. Typically, this will be a single year's graduates (e.g., members of the graduating class of 1993-1994). Potential interpretive problems emerge immediately:

• At the high school level, a class of graduates will include students who completed their vocational coursework at different times during their junior and senior years. This means that their qualification for, and possible participation in, training-related employment occurs at different times. This recognition, in turn, clouds the interpretation of a single post-schooling snapshot of employment status.
  
  An alternative approach that eliminates this alignment problem is to identify the reference population on the basis of when they completed their program of vocational courses. This is a common practice in states where the governance of vocational education is separated from other components of the state's public education system. Ignoring, for now, the difficulty of agreeing upon a practical definition of vocational program completion, a different interpretive problem arises when this approach is adopted: Since vocational program completion and graduation are treated as independent events, the effect of graduation on the observed measure of subsequent employment status might be overlooked.
  
  A solution to this problem is to account for both the timing of vocational program completion and the timing of graduation. Two other cautions should be heeded. As mentioned, vocational program completion and graduation are not always paired; graduation cannot be assumed to have occurred without explicit documentation of the event. The growth of modules of vocational courses, which fall between single course designations and well-known vocational program classifications, is threatening our ability to define a practical number of vocational content categories for performance measurement purposes.
  
  
• At the postsecondary level, many patterns of open-entry/open-exit are found, which increases the difficulty of aligning the timing of school leaving with a single snapshot of subsequent employment status. An example of how to respond to this challenge is provided in the final section of this guide.

1. Clearly identify the reference population, so no ambiguity arises about who is included. It is assumed here that the desired reference population of former students can be identified by their social security numbers. Confidence in the accuracy of this assumption is increasing as time passes because changes in reporting requirements for tax and educational accountability purposes have combined in recent years to reduce the percentage of students who cannot be identified in this way. However, important omissions persist. Recent immigrants to the U.S. are unlikely to obtain a social security number right away. Many of these immigrants enroll in community college classes to improve their language and employability skills. This presents a classic measurement error problem, since the very first step in Figure 2 cannot be taken for such enrollees.
   
   Electronic reporting has simultaneously increased the accuracy of reporting, which is important because an error at this point makes it impossible to create an accurate longitudinal record. The appearance of student social security numbers on school records cannot be assumed to mean that record matching can be carried out. Ohio's schools maintain student records that include social security number identifiers, but school officials are prohibited from transmitting this information to the state for accountability purposes. No attempt is made here to offer a generic statement about what is permissible in a particular jurisdiction. Each reader who is concerned about this topic is urged to investigate applicable laws and administrative policies for their own situation. Pfeiffer (1994) and Levesque and Alt (1994) offer pertinent background information.
   
   
2. Be sure that all members of this population are accounted for in each subsequent tabulation of recorded employment status. The most frequent criticism of traditional vocational education follow-up practices is that nonresponse biases diminish, and perhaps destroy, the reliability of alleged outcomes. It will become clear in subsequent sections of this guide that the use of state employment security agency records does not overcome this problem. However, an extraordinary innovation in interstate data exchange capability would follow favorable Congressional action on the anticipated recommendation by the Bureau of Labor Statistics that a national distributed database capability be established.

1. The top oval includes all of the former students who appear in the particular state employment security agency's records of reported employment during the July/August/September quarter of 1991; thus the designation 1991:3.
   
   For most high school students, the third quarter (July-September) can be assumed to be the first, or transition, quarter following graduation. Those who leave school without a diploma or who must attend summer school to complete their graduation requirements are exceptions to this general rule. Care must also be exercised to account for the timing of enrollment in postsecondary coursework, which usually begins in August or September following graduation; that is, during the third quarter. Anecdotal observation of international and domestic travel sojourns by recent high school graduates suggests that the third quarter of the year of graduation is unlikely to be representative of post-graduation employment status. The primary value of this snapshot of employment status is to identify those who continue a previously established affiliation with a particular employer, as well as those who immediately accept a job with a new employer. The distribution of school leaving dates for community college students is less concentrated, so more care must be exercised in defining a particular quarter as the transition quarter.
   
   
2. The middle oval in the vertical stack of three in Figure 2 includes those former students for whom no record of 1991:3 employment was found, but who did appear in a later quarter's records. The homogeneity of this population depends upon the length of the observation period; that is, how many quarterly snapshots of a former student's status have been carried out to determine whether they were employed in the same state.
   
   Currently, most states that are known to be using state employment security agency administrative records to document post-schooling employment status record a single quarter's status. The reference quarter differs, with the October-December quarter of the year of school-leaving and the January-March quarter of the next year as the most popular choices. Florida's Education and Training Placement Information Program (FETPIP) has created longitudinal files that include multiple annual observations of the October-December quarter for particular reference populations of former students. Similarly, Washington's State Board for Community & Technical Colleges acquires multiple annual snapshots of the January-March quarter for its reference populations of former community college students.
   
   
3. The bottom oval in the vertical stack of three, which is labeled no post-school employment recorded through 1992:4, includes those members of the reference population of former students who were not found in any of the six quarterly snapshots of reported employment covering the period 1991:3 through 1992:4; that is, July 1991 through December 1992.
   
   The location and status of the former students represented in this oval is unknown. They may be working in another state. They may be a federal government civilian employee or member of the military. They may work for the U.S. Postal Service, for a railroad, or for a religious or philanthropic organization. They may be self-employed or working as a commissions-only independent contractor. They may be enrolled in a public or private postsecondary school in this state or elsewhere. They may be incarcerated or hospitalized. They may be traveling internationally or domestically.
   
   The point of the previous paragraph is to acknowledge that attention can be focused on the members of the reference population who appear in the top or middle ovals (those who are found in the state employment security agency's database) or on those who end up in the bottom, residual oval. Those who tout the value of the employment security agency's data will emphasize what can be documented, while detractors will highlight what cannot be documented.

1. The top of the seven oval stack is of particular importance. It represents a direct challenge to the accuracy of the placement concept. Each of the former students who is included in this category of post-schooling employment status has been reported as working for the same employer during each of the four quarters of the year that they left school.
   
   For virtually all former high school students, appearance in this employment category means that they continued to work for the same employer for between four and six months while they were still in school, and for between four and six months after graduation. The four-month intervals would apply if a high school senior was paid for employment during any part of March, then remained employed from then through at least the beginning of October. This would result in recorded employment during each of the four quarters of the year, which actually represents concurrent employment and high school enrollment from the beginning date of employment in March through the graduation date, followed by employment only from then through the date in October when this employment affiliation ended. Verification of the employment only status would require matching of this former student's social security number with available postsecondary enrollment records.
   
   The maximum pairing of two six-month employment spells would occur only if the student was employed at the beginning of January and then continued that affiliation through December. Here, use of the word maximum refers to the longest possible length of continuous employment during the one year reference period. This is an arbitrary choice. The reference period can be lengthened by beginning earlier or monitoring longer after a student's graduation. The recording of earlier employment status might be of particular importance when the reference population is former community college students, many of whom have years of previous and concurrent employment experience. More diagnostics are required to accurately identify continuity of employment affiliation for former students in postsecondary programs because multiple exit dates are possible.
   
   
2. The second and third ovals (from the top in Figure 2) are separated for a reason that is not apparent from this figure alone. Former students represented in each of these two ovals had been reported as employed by at least one different employer during the April-June and July-September quarters of the year they left school. What distinguishes the two groups is that the former students in the upper of the two ovals had also been reported as employed in the January-March quarter, while those in the lower of the two ovals had not been reported as employed during this three-month period. The "no 1991:1 earnings reported" label indicates that these former students were not found in the first quarter records of employment reported to the state's employment security agency. The finding applies to the other two pairings of ovals in the stack of seven in Figure 2. The first quarter is thought to be of particular importance because it allows the investigator to distinguish between cases in which employment may have been reported during some part of June, following graduation but still during the second quarter, and those cases in which genuine continuity of an already established employer affiliation occurs.
   
   
3. The former students who are classified in the fourth and fifth ovals (from the top in Figure 2) are all characterized by the uniform descriptor that they did not appear in the state employment security agency's database of reported employment during the July-September quarter of the year they left school. In other words, there was a distinct break between their leaving school and the appearance of their first reported post-schooling employment affiliation. Again, the former students who exhibit this common characteristic have then been split into those who had been reported as employed at some time during the January-March quarter, and those who had not been reported as employed during this three-month period.
   
   
4. The bottom pairing of ovals in the stack of seven in Figure 2 covers those who had been reported as employed in the first quarter of the year they left school, but had not been reported as employed at any time from July 1 of that year through December 31 of the following year, which covers the first eighteen months after they left school. This assumes the former students all left school in June, which is a reasonable assumption in most high-school circumstances, but not for many postsecondary situations. It is possible for the former students found in either of these paired ovals to have been reported as employed during the April-June quarter of the year they left school. This possibility is downplayed here only because attention is later focused on comparisons of those who sustain preexisting employer affiliations and those who start anew. One isolated spell of reported employment is of little consequence in such inquiries. For other purposes, awareness of this employment may be important.
   
   From a vocational education standpoint the former students who are classified in the bottom oval in the stack of seven in Figure 2 are of particular interest. Assume that the reference population is high school graduates in a particular year, all of whom had completed some type of vocational program. A finding that none of these students had been reported as employed in the same state during the next eighteen months can at least be characterized as a signal to investigate further. Many explanations that are consistent with vocational education's mission will apply to some members of this group of former high school students. Some will have left the state to accept jobs that utilize the skills they have acquired. Others will have enrolled in postsecondary programs to build upon the foundation of skills they have already learned. Still others will be working in one of the noncovered types of employment (such as federal government civilian or military employment, self-employment, work for a railroad, and affiliation with a philanthropic or religious organization).
   
   This unknown status is particularly important when comparisons among populations of former students are anticipated. Different expectations of post-schooling behavior are likely to emerge such as the probability of postsecondary enrollment for a class of high school graduates. The pertinent management decision that must be made is whether, and how far, to pursue the classification of these unknowns. Some components of this classification can be accomplished in a routine manner at relatively low cost. Exchanges of information between secondary and postsecondary levels within a state are occurring more frequently. These voluntary administrative actions reduce the number of unknown cases and increase the public's understanding of, and confidence in, reported outcomes.|

Up to this point, a conceptual foundation has been laid for documenting employment outcomes from investments in vocational education, and practical categories of previous, concurrent, and post-schooling employment status have been identified. The next three subsections illustrate how this conceptual framework (Figure 1) and these employment categories (Figure 2) can be used to develop straightforward reports for management use. The single goal in these sections is to provide those who have an interest in local, state, or national vocational education activities with information that might reasonably be expected to affect management decisions, which, in turn, will increase vocational education's value to students and employers alike.

Employment Measures

Figure 3 fills in the conceptual shell from Figure 2 with the actual distribution of employment status for two populations of former students. Members of both populations graduated from a public high school in the same state during the 1990-1991 school year. The left side of the figure covers vocational program completers, and the right side covers nonvocational graduates. These reference groups were chosen because there is keen interest in the comparative employment outcomes for members of these two populations. In addition, this particular comparison reveals some relationships that illustrate conceptual points made earlier.

This state certifies vocational programs, but the content of a particular vocational program classification is not necessarily uniform across approved programs in different public high schools. The absence of content homogeneity increases the importance of data elements that identify differences among programs that are classified together at a higher level of aggregation. Often, the desired data elements are not available in statewide databases. There is an urgent need to identify what these data elements are, to establish priorities for introducing them, and to work with local and state management information specialists to accomplish this through informed voluntary action.

Remember that each of the three- and seven-oval stacks in Figure 3 sums to the respective total number of graduates; all members of each population are accounted for. The employment counts in the top oval of each stack provide a direct comparison of the percentage of former students who exhibit a continuous employer affiliation through all four quarters of the year they graduated from high school. These former students cannot be said to have been placed with this employer, at least not in a post-schooling sense.

Awareness of the higher rate of continuous employer affiliation for the vocational program completers, when compared to their nonvocational classmates, may be interpreted as good or bad news by a program's supporters and critics. Those who see this as good news might contend that the continuity of affiliation indicates that the employer values the employee's productivity that has been achieved through a combination of work-site and school-based learning. Those who interpret this higher rate as bad news may counter that the continuity could signal an inability to move from temporary after-school employment to a more meaningful first step onto a career ladder. Additional information must be examined to distinguish between these views. Some diagnostics of this type appear later in this guide. This impasse, based on one snapshot of a former student's employment status, strengthens a point made earlier. Multiple observations of a former student's status, and multiple
descriptors of each of these, are required to provide reports that have high management value; that is, that might actually affect a decision.

Approximately one out of every four of the former students in each of these high school graduate populations was reported as working in both the second and third quarters of this graduation year, but for a different employer. This might be characterized as what many observers think of as the typical transition situation for a high school graduate in the U.S. today.

It is interesting to speculate why this pattern is thought to be typical when only one out of every four cases satisfies this employment mobility criterion. The observed range of percentages for two reference groups in two states (four cells) is from a low of 25% to a high of 27%, and this range is stable across different years of high school graduation.

The surprise in Figure 3 is the high percentage of vocational program completers who did not appear in the state's employment and earnings records during the twenty-four month pre-/post-observation period (January 1991-December 1992). This figure (29%) is more than twice as high as the percentage for a population of 1990-1991 high school graduates who completed a vocational program in another state (12%).

Figures 4 and 5 reveal very different comparative rates of unknown status for community college students who completed vocational and nonvocational programs respectively. This difference is discussed at that point, but it is important to note here that the educational level is a critical factor in accounting for the rate of unknown status cases that appear. Since these are the types of figures that are likely to be extracted from a carefully documented report, and then repeated out of context, extreme care must be exercised to assure reader awareness of the investigator's own explanation for comparative results. One reason why state employment security agencies have not been deluged with requests for access to their administrative records is that many third-parties fear the unknown. Cautious vocational education administrators have exhibited a pervasive leeriness of establishing an outcomes measurement system that they cannot control.

The unit of analysis in Figure 3 is a statewide class of high school graduates who had completed either a vocational or nonvocational curriculum. Any other unit of analysis can be adopted using the same basic shell first introduced in Figure 2. Vocational/ nonvocational comparisons can be prepared for each school district within a state, and these can then be compared across districts. Selected pairs of vocational programs can be compared at the local, district, or statewide levels. Individual schools can be compared within a single district.

When a particular unit of analysis is chosen, an investigator is obliged to think through how the substitution of a different unit of analysis might affect the interpretation of observed employment status distributions. Alertness to the possible occurrence of small cell sizes and assurance of strict compliance with confidentiality requirements are of paramount importance (see Stevens, 1994b, 1994e).

Two recommendations emerge from this examination of Figure 3:

1. When a finding is counterintuitive, double-check the calculation, and then seek some basis for comparison with what is thought to be an appropriate comparison group (e.g., the same reference population in another state). This will become increasingly feasible as more states adopt longitudinal reporting practices.
   
   
2. Always conduct diagnostics with respect to noncovered employment possibilities. This is particularly important when a substate jurisdiction is pertinent such as when a metropolitan school district lies on a state's border with another state. The importance of federal government civilian and military employment opportunities in the jurisdiction should be considered, as should unusual patterns of self-employment. In many cases, these diagnostics can be carried out as mind-experiments or conceptual exercises without actually incurring the costs to collect pertinent employment data. Expert advice can be solicited from local education authorities and from the state employment security agency.

Earnings Concepts

The Content of a Wage Record Revisited

A small number of states, including Florida, Oregon, and Washington, require employers to report the number of hours or weeks each employee worked during the reference quarter. The accuracy of this time-unit information depends on (1) the ability of a reporting employer, or their agent, to provide the desired information; (2) their motivation to attempt to provide accurate information; and (3) the receiving agency's quality control procedures. These vary from state to state. Washington's unemployment compensation law includes hours of work as a factor in their tax computation, so there is a reciprocal interest by the reporting and receiving parties to pursue quality information. Recently, Oregon switched from a weeks-of-work reporting requirement to an hours requirement. This was done in part to satisfy third-party users who have long sought an ability to derive an hourly wage rate equivalent from a quarterly earnings figures and in part out of recognition that employers do not routinely maintain a weeks-of-work data element in their personnel files. Florida's employment security agency is currently reviewing many aspects of the state's unemployment compensation law, including advocacy for dropping the required reporting of a weeks-of-work figure.

Extreme caution must be exercised when a time-unit measure is used to derive a synthetic hourly wage equivalent from a reported quarterly earnings amount. Employer payments of end-of-year bonuses and other types of compensation that are distributed unevenly throughout a year may bear little or no relationship to the number of hours that were worked during the reference period. When longitudinal tracking of adult earnings may be relevant, a higher level of caution is urged. Terminated employees sometimes receive substantial early-retirement lump-sum payments, the cash equivalent of unused benefits, and other one-time payments that occur in one or more quarters after the former employee has left the business.

The definition of reportable earnings is codified in each state's unemployment compensation law. There are definitional differences in these state-specific laws. Investigators are encouraged to include the particular definition that applies when wage records are used for vocational education follow-up purposes.

Reference to an employer identifier masks a number of technical issues associated with the identity of a particular reporting entity. State employment security agency personnel refer to reporting units, not employers. A reporting unit can be a single business, one establishment in a multi-establishment enterprise, or a group of business entities that have received permission to be treated as a single entity. Each business that is covered by a state's unemployment compensation law has both a Federal Employer Identification Number (FEIN) and a state-specific unemployment compensation account number. In most states, a multi-establishment business entity can choose to submit its quarterly wage reports under a single umbrella identifier, or separately for each establishment. This option should not be confused with a state's participation in the Business Establishment List (BEL) program of the Bureau of Labor Statistics, which requires multi-establishment businesses to report how many, but not which, employees work at each establishment location.

It is also important to understand that the Bureau of Labor Statistics-State Employment Security Agency cooperative program commonly known as the ES-202 program, based on a long obsolete paper form number, asks employers to report the average number of employees who were paid for employment during the pay period that includes the twelfth day of the month. This contrasts with the transmittal of quarterly wage reports for all employees who were paid during the reference quarter.

Many of the quarterly reports submitted to a state employment security agency are now prepared by service bureaus that process compensation data for multiple employers. Most states allow an employer, or their agent, to use any address of record on these quarterly reports such as a headquarter's address, an accounting firm or legal counsel's office location, or a service bureau's address. This means that extreme care must be exercised in describing where former students are employed in a state.

Multi-establishment businesses often have more than one Standard Industrial Classification (SIC) code assigned to their business activities. When a new business requests an unemployment compensation account number for the first time, a questionnaire is given to them asking for a description of the business' major activity. The state employment security agency's research unit assigns a four-digit SIC code based on the information that is provided. The accuracy of this code is then reviewed on a three-year cycle as part of another Bureau of Labor Statistics-State Employment Security Agency cooperative program. Mergers and acquisitions can affect the accuracy of a business' SIC code until the next review cycle. In any case, it may be difficult, or even impossible, to associate a former student's employment with a unique industrial affiliation.

The relatively new and growing phenomenon of employee leasing has challenged state employment security agencies in their ongoing attempt to sort out distinctions between the reporting entity itself and where people actually work. For different reasons, a state employment security agency is interested in knowing about a particular employee's tie to a leasing agent and to the work site. It is not very informative to know that a leasing company employs 7,500 people without also knowing that these are distributed across manufacturing, wholesale and retail trade, and service sector assignments.

Having urged caution, it is important to keep these warnings in proper perspective. Most reporting units are single establishment businesses whose location and industrial code are both known, and whose use of a payroll vendor or leasing agent can easily be identified.

Multiple Wage Records Within and Across Quarters

1. One wage record might be designated as the primary record for this reference quarter.
   
   
2. The reported earnings amounts that appear in each of the records can be added together.

When one quarter is designated as the reference period for documenting a former student's earnings, and no other data is requested from the state employment security agency, then little can be done to reduce the types of ambiguity that have been described here. However, if a continuous longitudinal record can be created for two or more sequential quarters, more can be said about a former student's employment affiliations and earnings.

Various types of diagnostics can be carried out:

1. Perform a quarter-to-quarter comparison of a former student's employer affiliations to reveal the likely sequence of events. If one employer identifier appears in two sequential quarters, while another employer identifier appears in only the first of these two quarters, then it is reasonable to assume that the former student moved from one employer to another.
   
   It remains possible in this case that the former student had been moonlighting during the first quarter, holding two jobs at the same time, but then quit the second job before the end of the first quarter. The conclusion that the former student had moved from one job to another would be inaccurate. The authors have conducted extensive diagnostics using three-quarter sequences, which permit the investigator to determine with substantial confidence whether an employee was employed throughout the reference quarter. The procedure followed is to first compare employer identifiers during the first and second of the three quarters. If a match of employer identifiers is found, then it is concluded that the employee was working for this employer at the beginning of the second quarter. The comparison of employer identifiers is then repeated for the second and third quarters. Now if a match occurs it is concluded that the employee was working for the employer at the end of the second quarter. So, through these two steps, it has been determined with a high probability of accuracy that the former student worked for this employer throughout the second quarter. It is still possible that a recurring pattern of intermittent employment during these three months has been overlooked.
   
   
2. Conduct quarter-to-quarter comparisons of earnings levels to determine the stability of a former student's earnings. This step is particularly important if first or fourth quarter earnings are being used, since these are the most likely times for the payment of annual bonuses. This step will be less important when the reference population is former students in high school vocational education courses because relatively few of them might be expected to be eligible to receive compensation in the form of a bonus.
   
   
3. Establish a federal minimum wage full-quarter/full-time equivalent earnings floor to determine whether a former student's earnings reported by any one employer during the reference quarters fall above or below this threshold. Readers who are old enough to remember when most adults who worked were employed full-time year-round are warned to expect a high percentage of cases that fall below this threshold, particularly when the reference population is former high school students. Again, matches with postsecondary enrollment records can provide some indication of the probability that the observed earnings level is associated with part-time employment.
   
   
4. Add a criterion that a former student appear in each of four sequential quarters of the wage records database. This then becomes the equivalent of year-round employment. This procedure is recommended when an investigator intends to prepare a public release of information about vocational education outcomes. The first, and perhaps only, information many nonspecialists want is an answer to the question, "How much are graduates earning if they work full-time year-round?" This recommendation is not intended to downplay the importance of informing the public about the incidence and geographic/demographic correlates of cases when this criterion is not met.

Earnings Measures

Currently, this state's management information system does not include a data element that identifies the year/month or term of completion. Investigators who intend to replicate or refine this approach are urged to attempt to acquire both the year/month of completion of a vocational program and a data element that indicates whether a completer also graduated. This information is needed to investigate the independent effects on earnings of credits, program completion, and receipt of a credential (see Kane & Rouse, 1995a). The concepts incorporated in Figure 1 are pertinent here. Previous and concurrent work experience and other educational achievements must be considered if any attribution of outcomes is intended.

The average earnings amounts that appear in Figure 4 reflect an uncensored mix of full- and part-time employment during all or part of the reference quarter. If a member of the reference population had any reported earnings, no matter how small the amount, they
were included in the calculation of average quarterly earnings figures. The final section includes examples of censored subgroups, which apply different criteria for inclusion such as an earnings floor equal to the full-time/full-quarter federal minimum wage level and presence of at least one wage record in each of the four quarters of the reference year.

The highest average quarterly earnings level in Figure 4 is for former students who were already employed by this employer before they left school. This is a typical situation in which the observed earnings level cannot be described as a community college outcome alone.

At a minimum, based only on the information that is provided in Figure 4, the observed earnings level reflects the combined effects of the community college exposure and the concurrent work experience. Almost half of the reference population of former community college students falls into this category, so the importance of avoiding reliance on a single snapshot of post-schooling earnings should be apparent. The rapid growth of adult enrollments, which have resulted in a wide range of transcript patterns, including mixes of credit and noncredit course taking, has increased the difficulty of isolating the net impact of the current spell of community college exposure on observed earnings. Similarly, less-than-full-time enrollment is now the norm, and varied stop-out profiles occur. Together, these features of today's community college environment severely limit the relevance of single snapshot evidence of post-schooling earnings.

The weakness of single snapshot post-schooling earnings as a metric of community college outcome also is seen in the two earnings levels that appear in the lower right corner of Figure 4. Members of each of these reference groups completed a community college vocational program in one state during 1990-1991, did not have any reported earnings in that state during the July-September quarter of 1991, but did have reported earnings in the October-December quarter of 1991. What distinguishes the two groups is that members of one group had reported earnings during the January-March quarter of 1991, while members of the other group did not have such reported earnings. The average post-schooling earnings shown in Figure 4 for the latter group is only 52% of the former group's average. This difference would not be known if only a single snapshot of post-schooling earnings had been taken.

Figure 5 is the earnings equivalent of Figure 3. The left side of Figure 5 repeats the content of Figure 4, while the right side introduces new comparative information for nonvocational program completers. The statewide unit of analysis and 1990-1991 community college reference population continue unchanged. Each of the five paired vocational/nonvocational comparisons, based on post-schooling employment status, reveals a higher average earnings level for the vocational group.

Readers who are unfamiliar or uncomfortable with statistical terminology are urged to be particularly cautious in the use of state employment security agency earnings records. Mean earnings amounts are reported here. Some investigators use median earnings because they want to avoid the effect of outlier values on the average that is to be reported. Standard error figures are provided here to provide a basis for deciding whether an observed difference between two means can be said to be statistically significant. Confusion frequently arises about the difference between statistical significance and substantive importance. An observed difference between a pair of reported average earnings figures may fall in any one of the four cells of a significance-importance matrix.

The earnings amounts that appear in Figures 4 and 5 have been adjusted using a Gross Domestic Product Implicit Deflator Series factor. This allows pre- and post-schooling earnings levels to be compared in real rather than nominal terms--that is, after removing changes in earnings over time that are considered to be attributable to a general increase in prices rather than to increased value of the former student.

The previous two paragraphs, which use a terminology that may be unfamiliar to some readers, highlights a point that is of critical importance to the vocational education community. A proper balance must be found between technical accuracy of accountability reports and their transparency or readability.

Mobility and Its Consequences

can be viewed as a complement to, or as a substitute for, on-the-job training. The relevance of this distinction can be understood more easily if entry-level jobs and opportunities that are more demanding of skill competency are treated separately.

Entry-Level Employment Opportunities

Using Figure 1 as a framework for collecting and analyzing data, an investigator should be able to assemble a database of candidate qualification factors, employer requirement factors, and job opportunity descriptors that can identify which bundles of candidate qualifications are actually matched with what bundles of employer requirements. A match occurs when a new hire is detected.

Mid-Level and Advanced Opportunities

Implications for Vocational Education Outcomes Measurement

There has been a distinct shift in vocational education's role in recent years. One-time exposure to school-based entry-level skill development has diminished in importance, while repeated participation in specialized learning has grown. The either-or context of skill training in past decades (either the school provided the skills or the employers provided the skills) has been supplanted by the and context of the 1990s (workplace training and complementary recurring school-based continuing education). Vocational education's reporting practices, management information systems, and management decisions must be aligned with this reality.

It is no longer practical to ignore the importance of concurrent employment and school enrollment. At the high school level, states are struggling to identify and build upon evidence of concurrent education and work activities that lead to rewarding opportunities after graduation; often including a continuing affiliation with the same employer combined with selective course taking at the postsecondary level. At the community college level, the struggle shifts to ways to document the value-added that arises from diverse enrollment patterns that are not revealed in management information systems and standard reports that were designed in a linear world of transfer students and a few well-defined occupational programs. Advances on either front, high school or community college, will require a conscious downplaying of single snapshot performance measures. More relevant, but not necessarily more expensive, management information systems must be devised. Each data element in such a system should be justified in terms of its potential contribution to affecting a decision. This is the criterion for judging mobility data in the next subsection.

Measurement and Interpretation of Mobility

Progress toward sorting out these situations, which have very different management implications, can be made by combining and analyzing data elements currently available in state employment security agency databases. Figure 5 represents a first step toward understanding whether, and how, awareness of employee mobility patterns can be used to refine vocational education performance measurement practices.

Figure 6 is based on the documented mobility patterns of 2,260 former high school students who completed a vocational program and graduated in 1986. Here, stayers are defined as those who were reported as working for the same employer in each of the four reference periods (1986:3 or 1986:4 after graduating from high school; 1987-1988, 1989-1990, and 1991-1992). These stayers must have been reported as working for this employer during 1992:4 to be classified in this way. Movers are those who did not satisfy the criteria for being designated as a stayer, but were reported as working for some other employer in the state during each of the reference periods.

These students represent twenty-eight school districts in one state, with 90% of the records having been drawn from five of these districts. Furthermore, since the intent is to illustrate mobility patterns, each of these former students must have appeared in the state's wage record database in 1986 and 1992. Any member of this population of former students who graduated in 1986 may have left the state, as long as they returned to appear in the 1992 wage record file. Those who are reported as stayers in each of the four two-year snapshot intervals must have satisfied the more stringent criterion that they were reported as working for the same employer in each of the four snapshots. It is possible, but unlikely, that a former student who meets this requirement left the state and was employed elsewhere one or more times. For example, regular seasonal employment in another climate zone could have occurred. This is mentioned because the ultimate goal of this type of analysis is to isolate the relative contribution of school-based education and work-site training on employment status and earnings. If someone who is identified here as a stayer was, in fact, attending school or working out of state, then these events would be missed and the interpretation of the available data would be affected.

Only 8% of the former high-school students who are known to have been employed in the same state in 1986 and 1992 were found to have stayed with the same employer throughout this six and one-half year post-schooling period. The comparable stayer rate for completers of a community college vocational program in two of this state's community college districts, all of whom meet the same classification criteria, is 29%. Just over one-third of the former high school students moved in each of the three employment affiliation
comparisons. Others fall into mixed time sequences of being classified as a stayer or a mover.

Figure 6 illustrates a first step toward developing information that will have practical management value. The frequency of observation (i.e., the snapshot interval) can be increased to cover just one year, or even each quarter, following a student's departure from the reference educational activity. The duration of observation can be extended by periodically updating the database. Differences in mobility patterns among districts, across vocational programs, between vocational and nonvocational programs, and among levels of education can be documented. This assertion is accurate only when minimum cell-size thresholds are satisfied. Even when this criterion is met, caution must be exercised to be sure that the reported mobility pattern itself does not reveal the identity of a particular former student.

Figure 7 represents a second step toward developing insights that are of practical management value. Here, the same basic setup that was used in Figure 6 is repeated, except that now averages of reported earnings are displayed for stayers and movers. This presentation is based on former students who completed a community college vocational program in 1985-1986. Unlike Figures 4 and 5, each of which presented uncensored average quarterly earnings, Figure 7 displays uncensored average annual earnings levels. For each of the 779 former students covered in Figure 7, all reported earnings by any employer in the state during all four quarters of 1986, 1988, 1990, and 1992 are included. The benchmark annual earnings figure for the first reference interval of 1985-1986 would be expected to include part-time employment for a portion of the year for some of the former students. A quarter-to-quarter comparison of earnings for each of the quarters 1985:4 through 1986:3 would be expected to reveal this pattern.

The introduction of a threshold level of annual earnings equal to the federal minimum wage multiplied by full-time year-round employment would cut off the earnings distribution tail falling below $8,500, which would increase the reported averages across the board. The continuous stayers are more likely to have been employed full-time year-round throughout the observation period than their classmates who were classified as movers in each of the three comparisons of employer affiliation.

Figure 7 shows that at the end of 1986 those former students who are now known to have continued their employer affiliation for at least the next two years, already enjoyed a 35% annual earnings advantage over their classmates who are now known to have changed employers at least once during the same two-year period. For those who remained in these respective classifications for the next four years, this wedge of higher earnings favoring the stayers increased to 105%.

The patterns revealed in Figures 6 and 7 challenge those who are dedicated to the accurate measurement of vocational education outcomes. Clearly, diagnostics using additional data elements should be carried out to extract refined findings that can guide management actions in response to this evidence of disparity. Historically, in the United States, voluntary mobility has been associated with improvement of circumstances. Here, therefore, there is disquieting evidence that at least some of the observed mobility is likely to have been involuntary. If so, it is important to know whether the affected former students have failed to prosper for personal reasons or because their education failed to position them for competitive candidacy. Using the terms introduced in Figure 1, which elements in their overall bundle of qualifications contributed most to their downfall? Descriptors of educational attainment or descriptors of personal behavior or other factors? These are the kinds of diagnostics that promise to offer real value-added in support of those who must make decisions about their own career or the careers of others. These are feasible diagnostics with minor refinements of current information systems. Readers are referred to Klerman and Karoly (1994, pp. 31-48) for complementary evidence drawn from the National Longitudinal Survey of Youth (NLSY). This article is recommended because it requires readers to be alert to the nuances of constructing appropriate measures of transition from available data elements. Similar reader awareness is required in the final section of this guide, where ways to align the timing of school-leaving and a particular employer affiliation are described.

Up to this point, the fundamentals of documenting employment and earnings outcomes in vocational education have been developed. The second section provided a new conceptual basis for designing the data collection and analysis steps. Figure 1 introduced three sets of building-blocks: (1) candidate qualification factors, (2) employer requirement factors, and (3) the employment opportunities set. Each of these can be thought of as a vector, or list, of descriptors. An attempt can then be made to link each of these descriptors to one or more available data elements. This, in turn, can be followed by an exercise to
identify possible new data sources for descriptors that have no reliable data source currently. This wish list can then be priced and prioritized.

This third section has described a series of employment and earnings measures that reflect the interplay of the candidate qualification factors, employer requirement factors, and employment opportunity set. Reader alertness to the use of the word reflect here is essential. What many observers refer to as vocational education outcomes are actually a result of complex interdependencies and forces. The straightforward comparisons that have been presented in Figures 2 through 7 mask the lists of descriptors that should be considered before any attribution of cause-and-effect relationship is assigned. The next section returns to the components of Figure 1 and describes practical ways in which some of the interesting descriptors can be drawn from available data sources.