Assessing Students for Workplace Readiness

Phyllis T.H. Grummon, Institute for Research on Teaching Adults

Introduction

History and Policy

• national educational systems;
• national standards for educational performance;
  • specific methods for communicating the knowledge and skill needs of businesses to educational institutions and a responsiveness on the part of education for providing those skills;
  • widespread use of programs for ensuring that students have a smooth transition to the labor force; and
  • national workforce training and retraining systems.

Federal Legislation. Policy discussions and legislation have attempted to address some of the differences between the U.S. system of education and that of other nations. The Carl D. Perkins Vocational and Applied Technical Education Act of 1984, the School-to-Work Opportunities Act, and Goals 2000 each have components that address one or more of them. The Perkins Act, which focuses on vocational education and the integration of academic and vocational instruction, contains provisions requiring states and districts to develop performance standards and measures for evaluating the effectiveness of programs, not the skills acquisition of individual students. However, at least some states use this mandate to assess vocational education students on their workplace readiness skills, as well as on their preparation for specific jobs or occupations.

Goals 2000 was passed in 1994. This Act encourages broad educational reform, and includes a number of provisions affecting assessment of workplace readiness. It also includes provisions for the development of content and performance standards, but they have not yet been enacted. However, funding was provided to groups to develop content standards in a number of academic areas (e.g., English, social studies, science, history, art). The standards have been used by numerous groups to develop links between academic standards and workplace readiness standards, including the National Skills Standards Board, which was established by Goals 2000 specifically to work with industry, employers, labor organizations, employment trainers, and educational institutions to establish common skills standards.

The School-to-Work Opportunities Act was also passed in 1994. There is no explicit mandate in the Act for academic assessment beyond the expectation that students meet the academic content standards developed under Goals 2000. Members of the National Skills Standards Board have been appointed and have begun meeting to determine what standards should be adopted. Skills standards include: (1) a description of the segment of work for which the standard applies; (2) a list of essential knowledge and skills that are critical to the work segment; (3) a list of the essential tools and equipment critical to the work segment (if applicable); and (4) the criteria used to measure competency in performing the work segment (National Skills Standards Board, 1995). Standards will inform the development of skills certificates that can be used by students and others to convey to employers preparation in specific occupational areas.

The Perkins Act encourages the development of portable skills certificates as a means for increasing opportunities for technical preparation program participants. When developed, skills certificates will allow students to show their competence in occupations that do not have more traditional methods of assessing skills, and will be an important aspect of any attempt to assess workplace readiness.

Assessment Needs

A number of national taxonomies identify generic workplace readiness skills and knowledge. The most well known was developed in 1991 by the Secretary's Commission on Achieving Necessary Skills (SCANS). The Commission identified two levels of skills, foundation skills (e.g., reading, writing, mathematics, problem solving, personal) and workplace competencies (e.g., managing resources, understanding systems, working with others, using technology, acquiring and using information), which have served as the basis for the development of numerous workplace readiness programs, surveys, and curricula at the state and local level.

An earlier joint effort by the American Society for Training and Development and the U.S. Department of Labor resulted in a similar taxonomy that included skills sets in such areas as learning to learn, adaptability, group effectiveness, and reading, writing, and computation (Carnevale, Gainer, & Meltzer, 1990). Further, the Council of Chief State School Officers (1992 and 1995), through its Workplace Readiness Assessment Consortium, developed a Framework for Assessing Workplace Readiness Skills. The Framework includes skill descriptions in such areas as personal management, participation in work organizations, thinking/problem solving, communication, and workplace systems. Each of these taxonomies identifies similar generic workplace readiness skills, but the assessment of these skills has not been developed as quickly as their identification.

Assessment for specific occupations or jobs is much more common than for generic skills. National certifying examinations have been in existence for many years in a number of occupational areas. National organizations such as V-TECS (Vocational-Technical Education Consortium of the States) and NOCTI (National Occupational Competency Testing Institute) have helped many state and local programs develop assessments for specific occupations based on extensive task and job analyses. Processes such as DACUM (Developing a Curriculum), which are principally focused on curriculum development, also contribute to the identification of targeted skills and knowledge that frequently inform assessment practices for a school or district.

There is considerable discussion about whether generic workplace knowledge and skills can be assessed outside of the context of a specific occupational or content area. On the one hand, when asked what they are looking for in employees, employers tend to describe very general workplace skills, like the ability to get along with others. On the other hand, most people develop and use skills and knowledge best when they are focused in a specific occupational area. The question of the transferability of skills and knowledge--which is at the heart of the generic versus specific discussion--is one that has not been completely answered for either assessment or instruction. Some skills, like interpersonal skills, do seem to transfer. Others transfer only in part. For example, students may be able to read for meaning more easily in an occupational area of interest to them and be less able to read for meaning in a general subject area.

The push to integrate academic and vocational instruction has led to questions about how to assess students' academic performance within a vocational context and their vocational abilities (i.e., applied learning) within an academic context. The School-to-Work Opportunities Act addressed this issue by mandating that students meet the same academic content standards as those put forth in Goals 2000. Vocational students have had their knowledge of academic content assessed in relationship to specific occupational areas, but the testing ability of academically-achieving students to apply academic content to workplace settings has not been a focus of most state or local assessment systems. Thus, schools need to consider how to assess students' ability to apply academic knowledge and skills to the workplace.

Assessment of Generic Workplace Skills

The state of Michigan developed an Employability Skills Portfolio assessment system that allows students to document various workplace readiness skills by selecting the work that they believe best represents their ability to perform in each skill area. The Portfolio system includes benchmarks against which students and raters can score a portfolio's contents (Michigan Department of Education, 1993).

A self-assessment of workplace readiness skills, habits, and styles has been developed in a book called Working (Miles & Grummon, 1996). An advantage of a self-assessment is that it can cover more generic skills in a single instrument than can traditional tests, which need to be scored externally. Working allows students to rate themselves in such areas as taking responsibility, thinking in terms of systems, interest in lifelong learning, and adapting to change. While a self-assessment of SCANS skills cannot be used for hiring or selection, it can offer a teacher and students insights into students' perceived levels of these skills.

A number of organizations also report that they are developing banks of test items to assess students' SCANS or other generic workplace skills. For example, the SCANS Compendium III has created a bank that includes approximately 800 cognitive and affective items, covering eight areas, including: using technology, applying thinking skills, interpersonal skills, and maximizing resource allocations (Skillsnet Bulletin Board Posting, September 11,1996). Moreover, the U.S. Department of Labor and the U.S. Department of Education have jointly sponsored a National Job Analysis study through the ACT Center for Education and Work that is designed first to validate the SCANS competencies and then to develop assessments of them.

Types of Assessment

Selected Response. Teachers frequently use multiple choice, true-false, and matching tests to determine students' mastery of content area knowledge. They are known generically as selected response tests because students have to select the one (usually) best answer. Typically scored by someone other than the student, selected response tests can be constructed to have a very high degree of validity (they test what they say they do) and reliability (scores are stable over time for the same individual). By using enough items and trying them out on enough people, it is possible to construct a test that is a useful assessment of a student's content knowledge in a subject area. Certifying or licensing examinations in the health care field and the skilled trades, for example, have long used selected response methods to determine a student's readiness for a specific job. Using workplace subjects in content area tests is an easy way for teachers to help students learn how to apply academic content to workplace problems. Selected response tests are very useful for finding out if a student has the ability to identify facts or best practices in an area. Even such interpersonal subjects as teamwork or communication can have their knowledge components assessed through selected responses. The Work Keys assessment of teamwork uses such a strategy.

Constructed Response. Constructed response tests require students to elaborate in their answers. Essay tests are the most commonly used assessment of this type. Another type is evaluation of the accuracy or thoughtfulness of a student's calculations, as well as on the final answer. Scoring for constructed responses can be done either holistically or analytically. Holistic scoring requires that the scorer apply a set of criteria to the entire response, and then assign a single score based on the whole response. Analytic scoring also identifies a set of criteria or evaluative factors, but the scorer must assign points to each criterion and sum them for the total score. Constructed responses can be very useful in determining students' thoughts and reasons for answering as they did. More and more workplaces want employees who can learn and who know how to explore and solve problems. Constructed response assessments can help shed light on those abilities. The Michigan Department of Education (1993) has developed an assessment of students' ability to organize information that requires a constructed response. Students are given material on "Pat," Pat's abilities, and a job advertisement. They then must write to Pat to suggest ways to organize Pat's Employability Skills Portfolio so that Pat will be successful in applying for the job.

Performance-Based Assessments. Much has been written lately about performance-based or "authentic" assessments, which have been in use a long time, particularly in the workplace, for purposes of assessing skills and issuing licenses. Here, a student is rated by an observer, generally using a set of scoring criteria, while demonstrating a skill. The observer may score either holistically or analytically. There are several reasons for the increased interest in performance-based assessments (Popham, 1995). First, assessment using selected response or constructed response tests is seen as too limited; it may not allow students to fully demonstrate their higher-order thinking skills. Second, there is an increased awareness that students can leave school without being able to apply their content knowledge outside of the classroom, partly because tests do not ask students to demonstrate use of their knowledge or skills, but merely to recite them on selected response tests. Finally, there is concern that in high stakes testing teachers emphasize the information or strategies needed for the test so when tests are mostly selected response, teachers may not be helping students learn how to apply information. If assessments required performance or application of information, teachers would be more likely to include those activities in their classrooms.

The benefit of performance-based assessments is that they provide an actual behavior sample for judging a student's ability to use a skill or apply knowledge. Observation as an assessment tool is a drawback to these types of assessments, though; the tests take longer to administer and score than most other assessment types, and, when the results are to be used for making significant decisions about a student, multiple and well-trained observers are generally required to ensure the necessary levels of reliability for decision-making. It is also usually necessary to administer a fairly lengthy set of performance-based assessments (six or more) in order to get an accurate picture of the student's skills in an area (Shavelson & Baxter, 1992).

Portfolios. Portfolio assessment also has a long history in some professions and for some purposes. Artists, architects, and writers, for example, organize and present an array of their materials in a portfolio when seeking work. Portfolios, typically created over time, provide a way for students to collect the physical evidence demonstrating their acquisition and use of a skill or knowledge, and sometimes even to provide a rationale for the inclusion of the work. Portfolios provide an organized way for scorers to determine if the contents of the portfolio indicate that its owner has the skills and knowledge to meet a predetermined standard. Portfolios may be scored either holistically or analytically. As with performance-based assessment, the training of the scorers is critical to the reliability of the scoring process.

Portfolios have been used in a number of states to help students learn about their workplace readiness skills (e.g., Michigan, California). In a typical portfolio assessment process, teachers work with students to help them acquire the skills that workplaces are seeking, such as those exemplified by the SCANS taxonomies and to select examples of their work (in-school or out-of-school). Scorers review the portfolios against predetermined benchmarks and tell students whether their materials indicate proficiency in the skill. Portfolios of this type have generally been focused on developing students' skills, although students are encouraged to use them when applying for college or a job. The development of portfolios in a classroom represents an intersection between instruction and assessment that allows teachers to provide students with an additional level of feedback--how skills they are developing relate to skills needed in the workplace--without adding much additional instructional time. However, there can be considerable time devoted to rating portfolio contents against external standards or benchmarks, even for developmental purposes.

Matching Assessments with Purposes

To select an appropriate assessment a teacher must know how the information elicited from it will be used, since high stakes assessments used for placement decisions require much more rigor in their development to ensure high reliability and validity than do assessments used for low stakes purposes. Assessments designed to give students feedback for developmental purposes do not have to meet the same level of statistical or psychometric rigor that decision-making assessments do. Thus, any of the assessment types described above can be used effectively in classrooms for low stakes or medium stakes purposes. Teachers have considerable latitude in how they assess students' knowledge and skills, but they need to think about how assessment can contribute to students' learning about themselves, particularly in relationship to skills needed in the workplace.

Conclusion

This CenterFocus was developed at the Institute on Education and the Economy, Teachers College, Columbia University, which is part of the National Center for Research in Vocational Education (NCRVE).

References

Council of Chief State School Officers. (1992 and 1995). Framework for assessing workplace readiness skills. Washington, DC: Author.

Grummon, P. (1993). Assessing educational outcomes: Trends and opportunities. In T. Bynum, J. Cutcher-Gershenfeld, P. Grummon, B. Mullan, K. Roberts, & C. Weissert (Eds.), Policy choices: Framing the debate for Michigan's future. E. Lansing, MI: Michigan State University Press.

Michigan Department of Education. (1993). Employability skills assessment kit. Lansing, MI: Author.

Miles, C., & Grummon, P. (1996). Working. Clearwater, FL: H & H Publishing Company.

Popham, W.J. (1995). Classroom assessment: What teachers need to know. Boston, MA: Allyn and Bacon.

Saterfiel, T., & McLarty, J. (1995). Assessing employability skills. ERIC Digest. Greensboro, NC: ERIC Clearinghouse on Counseling and Student Services. (ED 391 109)

Secretary's Commission on Achieving Necessary Skills (SCANS). (1991). What work requires of schools: A SCANS report for America 2000. Washington, DC: U.S. Department of Labor.

Shavelson, R., & Baxter, G. (1992). What we've learned about assessing hands-on science. Educational Leadership, 49(8), 20-25.

Skillsnet. (1996 September 11). SCANS Compendium III Assessment Tools. Bulletin board posting.

This publication was published pursuant to a grant from the Office of Vocational and Adult Education, U.S. Department of Education, authorized by the Carl D. Perkins Vocational Education Act.