CenterFocus Logo

Assessing Students for Workplace Readiness

Centerfocus Number 15 / February 1997

Phyllis T.H. Grummon, Institute for Research on Teaching Adults

Introduction

Throughout this century, emphasis on the evaluation of students' achievement, knowledge, and skills has increased. Information from these evaluations has been used most often for determining students' readiness to advance in the educational system. Thus, students have gained little understanding of how their education will benefit their careers. More importantly, students and teachers do not have methods for translating students' academic achievement into a measurement of workplace readiness, and ultimately, success in a career. Assessing students for workplace readiness requires that schools know what skills and knowledge students need to succeed at work, and how to foster their acquisition.

History and Policy

Education for Work. Schools have always viewed the preparation of students for work as one of their roles. Throughout much of the century, though, training for specific jobs was assigned to vocational education, preparation for managerial and professional jobs fell to colleges, and general education was deemed sufficient for preparing the majority of students for their careers. Then, in the 1970s and 1980s, changes in world markets made it apparent that the country could not count on general education to provide students with sufficient preparation for most careers. Policymakers, business, and labor leaders, looking at the keys to competitiveness in other countries, saw that, along with other factors, the educational systems of global competitors provided them with a significant advantage over the U.S. (Grummon, 1993). Their educational systems included the following:

Federal Legislation. Policy discussions and legislation have attempted to address some of the differences between the U.S. system of education and that of other nations. The Carl D. Perkins Vocational and Applied Technical Education Act of 1984, the School-to-Work Opportunities Act, and Goals 2000 each have components that address one or more of them. The Perkins Act, which focuses on vocational education and the integration of academic and vocational instruction, contains provisions requiring states and districts to develop performance standards and measures for evaluating the effectiveness of programs, not the skills acquisition of individual students. However, at least some states use this mandate to assess vocational education students on their workplace readiness skills, as well as on their preparation for specific jobs or occupations.

Goals 2000 was passed in 1994. This Act encourages broad educational reform, and includes a number of provisions affecting assessment of workplace readiness. It also includes provisions for the development of content and performance standards, but they have not yet been enacted. However, funding was provided to groups to develop content standards in a number of academic areas (e.g., English, social studies, science, history, art). The standards have been used by numerous groups to develop links between academic standards and workplace readiness standards, including the National Skills Standards Board, which was established by Goals 2000 specifically to work with industry, employers, labor organizations, employment trainers, and educational institutions to establish common skills standards.

The School-to-Work Opportunities Act was also passed in 1994. There is no explicit mandate in the Act for academic assessment beyond the expectation that students meet the academic content standards developed under Goals 2000. Members of the National Skills Standards Board have been appointed and have begun meeting to determine what standards should be adopted. Skills standards include: (1) a description of the segment of work for which the standard applies; (2) a list of essential knowledge and skills that are critical to the work segment; (3) a list of the essential tools and equipment critical to the work segment (if applicable); and (4) the criteria used to measure competency in performing the work segment (National Skills Standards Board, 1995). Standards will inform the development of skills certificates that can be used by students and others to convey to employers preparation in specific occupational areas.

The Perkins Act encourages the development of portable skills certificates as a means for increasing opportunities for technical preparation program participants. When developed, skills certificates will allow students to show their competence in occupations that do not have more traditional methods of assessing skills, and will be an important aspect of any attempt to assess workplace readiness.

Assessment Needs

Identification of ways to assess workplace readiness both generally and for specific occupations can be one of the most significant undertakings of any school or teacher. Assessment for generic workplace readiness should start with a taxonomy of generic workplace skills. Assessment for specific occupations requires analysis of the skills and knowledge needed, which may also include generic skills within the context of a specific use.

A number of national taxonomies identify generic workplace readiness skills and knowledge. The most well known was developed in 1991 by the Secretary's Commission on Achieving Necessary Skills (SCANS). The Commission identified two levels of skills, foundation skills (e.g., reading, writing, mathematics, problem solving, personal) and workplace competencies (e.g., managing resources, understanding systems, working with others, using technology, acquiring and using information), which have served as the basis for the development of numerous workplace readiness programs, surveys, and curricula at the state and local level.

An earlier joint effort by the American Society for Training and Development and the U.S. Department of Labor resulted in a similar taxonomy that included skills sets in such areas as learning to learn, adaptability, group effectiveness, and reading, writing, and computation (Carnevale, Gainer, & Meltzer, 1990). Further, the Council of Chief State School Officers (1992 and 1995), through its Workplace Readiness Assessment Consortium, developed a Framework for Assessing Workplace Readiness Skills. The Framework includes skill descriptions in such areas as personal management, participation in work organizations, thinking/problem solving, communication, and workplace systems. Each of these taxonomies identifies similar generic workplace readiness skills, but the assessment of these skills has not been developed as quickly as their identification.

Assessment for specific occupations or jobs is much more common than for generic skills. National certifying examinations have been in existence for many years in a number of occupational areas. National organizations such as V-TECS (Vocational-Technical Education Consortium of the States) and NOCTI (National Occupational Competency Testing Institute) have helped many state and local programs develop assessments for specific occupations based on extensive task and job analyses. Processes such as DACUM (Developing a Curriculum), which are principally focused on curriculum development, also contribute to the identification of targeted skills and knowledge that frequently inform assessment practices for a school or district.

There is considerable discussion about whether generic workplace knowledge and skills can be assessed outside of the context of a specific occupational or content area. On the one hand, when asked what they are looking for in employees, employers tend to describe very general workplace skills, like the ability to get along with others. On the other hand, most people develop and use skills and knowledge best when they are focused in a specific occupational area. The question of the transferability of skills and knowledge--which is at the heart of the generic versus specific discussion--is one that has not been completely answered for either assessment or instruction. Some skills, like interpersonal skills, do seem to transfer. Others transfer only in part. For example, students may be able to read for meaning more easily in an occupational area of interest to them and be less able to read for meaning in a general subject area.

The push to integrate academic and vocational instruction has led to questions about how to assess students' academic performance within a vocational context and their vocational abilities (i.e., applied learning) within an academic context. The School-to-Work Opportunities Act addressed this issue by mandating that students meet the same academic content standards as those put forth in Goals 2000. Vocational students have had their knowledge of academic content assessed in relationship to specific occupational areas, but the testing ability of academically-achieving students to apply academic content to workplace settings has not been a focus of most state or local assessment systems. Thus, schools need to consider how to assess students' ability to apply academic knowledge and skills to the workplace.

Assessment of Generic Workplace Skills

Determining specific skills related to occupations or jobs is often more straightforward than trying to decide what generic workplace skills a student should acquire. The range of skills that cut across work settings makes it impossible for a single test to capture all of them. However, a number of organizations have developed assessments that focus on most of the commonly identified generic skills. ACT (formerly American College Testing), through its Work Keys program, has created a series of tests that assess skills common to a variety of workplaces. They cover subjects that can be easily taught and tested using traditional paper and pencil methods, and are linked with a series of job profiles that tie an individual's skill level with a specific job. Thus, companies whose jobs fit the established job profiles can use the results of a Work Keys assessment in their hiring and training processes (Saterfiel & McLarty, 1995).

The state of Michigan developed an Employability Skills Portfolio assessment system that allows students to document various workplace readiness skills by selecting the work that they believe best represents their ability to perform in each skill area. The Portfolio system includes benchmarks against which students and raters can score a portfolio's contents (Michigan Department of Education, 1993).

A self-assessment of workplace readiness skills, habits, and styles has been developed in a book called Working (Miles & Grummon, 1996). An advantage of a self-assessment is that it can cover more generic skills in a single instrument than can traditional tests, which need to be scored externally. Working allows students to rate themselves in such areas as taking responsibility, thinking in terms of systems, interest in lifelong learning, and adapting to change. While a self-assessment of SCANS skills cannot be used for hiring or selection, it can offer a teacher and students insights into students' perceived levels of these skills.

A number of organizations also report that they are developing banks of test items to assess students' SCANS or other generic workplace skills. For example, the SCANS Compendium III has created a bank that includes approximately 800 cognitive and affective items, covering eight areas, including: using technology, applying thinking skills, interpersonal skills, and maximizing resource allocations (Skillsnet Bulletin Board Posting, September 11,1996). Moreover, the U.S. Department of Labor and the U.S. Department of Education have jointly sponsored a National Job Analysis study through the ACT Center for Education and Work that is designed first to validate the SCANS competencies and then to develop assessments of them.

Types of Assessment

There are a variety of ways for students to discover their workplace-related skills and knowledge with the help of educators. It is important to understand, however, that no single assessment or assessment method can completely measure a student's range of skills and knowledge in a content area. Thus, it is necessary to use several of the types of assessment described below to help students learn about their skills in even a single content area. In addition, assessment methods should include some that are self-scored as well as those externally-scored or -rated. Students' ratings of their own skills and knowledge can be very valuable, and for some workplace areas, like career understanding, may be the most viable assessment method. Finally, students can be compared against a predetermined standard of success (criterion-referenced assessment) or can be compared against others who have taken the test (norm-referenced assessment). Again, the uses of the assessment should help determine whether a student must meet an absolute standard or a relative one. Here are some ways to assess workplace readiness skills and knowledge.

Selected Response. Teachers frequently use multiple choice, true-false, and matching tests to determine students' mastery of content area knowledge. They are known generically as selected response tests because students have to select the one (usually) best answer. Typically scored by someone other than the student, selected response tests can be constructed to have a very high degree of validity (they test what they say they do) and reliability (scores are stable over time for the same individual). By using enough items and trying them out on enough people, it is possible to construct a test that is a useful assessment of a student's content knowledge in a subject area. Certifying or licensing examinations in the health care field and the skilled trades, for example, have long used selected response methods to determine a student's readiness for a specific job. Using workplace subjects in content area tests is an easy way for teachers to help students learn how to apply academic content to workplace problems. Selected response tests are very useful for finding out if a student has the ability to identify facts or best practices in an area. Even such interpersonal subjects as teamwork or communication can have their knowledge components assessed through selected responses. The Work Keys assessment of teamwork uses such a strategy.

Constructed Response. Constructed response tests require students to elaborate in their answers. Essay tests are the most commonly used assessment of this type. Another type is evaluation of the accuracy or thoughtfulness of a student's calculations, as well as on the final answer. Scoring for constructed responses can be done either holistically or analytically. Holistic scoring requires that the scorer apply a set of criteria to the entire response, and then assign a single score based on the whole response. Analytic scoring also identifies a set of criteria or evaluative factors, but the scorer must assign points to each criterion and sum them for the total score. Constructed responses can be very useful in determining students' thoughts and reasons for answering as they did. More and more workplaces want employees who can learn and who know how to explore and solve problems. Constructed response assessments can help shed light on those abilities. The Michigan Department of Education (1993) has developed an assessment of students' ability to organize information that requires a constructed response. Students are given material on "Pat," Pat's abilities, and a job advertisement. They then must write to Pat to suggest ways to organize Pat's Employability Skills Portfolio so that Pat will be successful in applying for the job.

Performance-Based Assessments. Much has been written lately about performance-based or "authentic" assessments, which have been in use a long time, particularly in the workplace, for purposes of assessing skills and issuing licenses. Here, a student is rated by an observer, generally using a set of scoring criteria, while demonstrating a skill. The observer may score either holistically or analytically. There are several reasons for the increased interest in performance-based assessments (Popham, 1995). First, assessment using selected response or constructed response tests is seen as too limited; it may not allow students to fully demonstrate their higher-order thinking skills. Second, there is an increased awareness that students can leave school without being able to apply their content knowledge outside of the classroom, partly because tests do not ask students to demonstrate use of their knowledge or skills, but merely to recite them on selected response tests. Finally, there is concern that in high stakes testing teachers emphasize the information or strategies needed for the test so when tests are mostly selected response, teachers may not be helping students learn how to apply information. If assessments required performance or application of information, teachers would be more likely to include those activities in their classrooms.

The benefit of performance-based assessments is that they provide an actual behavior sample for judging a student's ability to use a skill or apply knowledge. Observation as an assessment tool is a drawback to these types of assessments, though; the tests take longer to administer and score than most other assessment types, and, when the results are to be used for making significant decisions about a student, multiple and well-trained observers are generally required to ensure the necessary levels of reliability for decision-making. It is also usually necessary to administer a fairly lengthy set of performance-based assessments (six or more) in order to get an accurate picture of the student's skills in an area (Shavelson & Baxter, 1992).

Portfolios. Portfolio assessment also has a long history in some professions and for some purposes. Artists, architects, and writers, for example, organize and present an array of their materials in a portfolio when seeking work. Portfolios, typically created over time, provide a way for students to collect the physical evidence demonstrating their acquisition and use of a skill or knowledge, and sometimes even to provide a rationale for the inclusion of the work. Portfolios provide an organized way for scorers to determine if the contents of the portfolio indicate that its owner has the skills and knowledge to meet a predetermined standard. Portfolios may be scored either holistically or analytically. As with performance-based assessment, the training of the scorers is critical to the reliability of the scoring process.

Portfolios have been used in a number of states to help students learn about their workplace readiness skills (e.g., Michigan, California). In a typical portfolio assessment process, teachers work with students to help them acquire the skills that workplaces are seeking, such as those exemplified by the SCANS taxonomies and to select examples of their work (in-school or out-of-school). Scorers review the portfolios against predetermined benchmarks and tell students whether their materials indicate proficiency in the skill. Portfolios of this type have generally been focused on developing students' skills, although students are encouraged to use them when applying for college or a job. The development of portfolios in a classroom represents an intersection between instruction and assessment that allows teachers to provide students with an additional level of feedback--how skills they are developing relate to skills needed in the workplace--without adding much additional instructional time. However, there can be considerable time devoted to rating portfolio contents against external standards or benchmarks, even for developmental purposes.

Matching Assessments with Purposes

As discussed, assessments can serve a variety of purposes. Those used to make significant determinations of a student's ability (i.e., graduation from high school) are considered high stakes assessments. Low stakes assessments typically do not affect a student's future as directly. Weekly spelling tests indicate whether a student needs more work on spelling, but will probably not be used as the sole determinant of promotion. An assessment that tells a district whether a new curriculum is producing desired learning outcomes may be high stakes for the school, but low stakes for the student. Somewhere in between high and low stakes tests is a continuum of assessments (medium stakes tests) that have some effects on students (like the mid-term examination), but are not the sole determinant of a placement decision.

To select an appropriate assessment a teacher must know how the information elicited from it will be used, since high stakes assessments used for placement decisions require much more rigor in their development to ensure high reliability and validity than do assessments used for low stakes purposes. Assessments designed to give students feedback for developmental purposes do not have to meet the same level of statistical or psychometric rigor that decision-making assessments do. Thus, any of the assessment types described above can be used effectively in classrooms for low stakes or medium stakes purposes. Teachers have considerable latitude in how they assess students' knowledge and skills, but they need to think about how assessment can contribute to students' learning about themselves, particularly in relationship to skills needed in the workplace.

Conclusion

Workplace readiness assessment helps students translate what they have learned in classrooms into the skills and knowledge needed in a variety of workplaces. Teachers are already covering many of those skills--writing, computation, teamwork, problem solving, and creative thinking. What teachers, school districts, and states have not been doing as effectively is giving students explicit feedback on how well they are prepared to enter and succeed in the workplace. The generic and specific skills and knowledge needed in workplaces must be integrated into the ongoing assessment practices of schools. Only by assuring that students understand what skills they have mastered and where they must improve can we help them become productive citizens of a global economy.

This CenterFocus was developed at the Institute on Education and the Economy, Teachers College, Columbia University, which is part of the National Center for Research in Vocational Education (NCRVE).

References

Carnevale, A., Gainer, L., & Meltzer, A. (1990). Workplace basics: The essential skills employers want. San Francisco, CA: Jossey-Bass.

Council of Chief State School Officers. (1992 and 1995). Framework for assessing workplace readiness skills. Washington, DC: Author.

Grummon, P. (1993). Assessing educational outcomes: Trends and opportunities. In T. Bynum, J. Cutcher-Gershenfeld, P. Grummon, B. Mullan, K. Roberts, & C. Weissert (Eds.), Policy choices: Framing the debate for Michigan's future. E. Lansing, MI: Michigan State University Press.

Michigan Department of Education. (1993). Employability skills assessment kit. Lansing, MI: Author.

Miles, C., & Grummon, P. (1996). Working. Clearwater, FL: H & H Publishing Company.

Popham, W.J. (1995). Classroom assessment: What teachers need to know. Boston, MA: Allyn and Bacon.

Saterfiel, T., & McLarty, J. (1995). Assessing employability skills. ERIC Digest. Greensboro, NC: ERIC Clearinghouse on Counseling and Student Services. (ED 391 109)

Secretary's Commission on Achieving Necessary Skills (SCANS). (1991). What work requires of schools: A SCANS report for America 2000. Washington, DC: U.S. Department of Labor.

Shavelson, R., & Baxter, G. (1992). What we've learned about assessing hands-on science. Educational Leadership, 49(8), 20-25.

Skillsnet. (1996 September 11). SCANS Compendium III Assessment Tools. Bulletin board posting.

This publication was published pursuant to a grant from the Office of Vocational and Adult Education, U.S. Department of Education, authorized by the Carl D. Perkins Vocational Education Act.



CENTERFOCUS

National Center for Research in
Vocational Education
University of California at Berkeley

Address all comments, questions, and
requests for additional copies to:

NCRVE
2030 Addison Street, Suite 500
Berkeley, CA 94720-1674

Our toll-free number is
800-(old phone deleted)


CenterFocus Archive | No: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20