A Voluntary System to Assess Vocational Skills and Improve Instruction
Policy Rationale and Goals: California's Career-Technical Assessment Program (C-TAP) began in 1990 and was developed by Far West Laboratory (FWL), now known as WestEd, under contract to the California Department of Education. Since 1990, four assessment components have been developed for C-TAP for each of five vocational program areas: [1] agriculture, [2] business, [3] health careers, [4] home economics, and [5] industrial technology education. The components include written scenarios, projects, portfolios, and, more recently, multiple-choice tests.
C-TAP
was originally proposed as a statewide performance-based certification system for vocational students. Under C-TAP, students were to be given the option of demonstrating competency for employment or advanced training based on their mastery of California Model Curriculum Standards (MCSs) for programs offered in California high schools and Regional Occupational Centers or Programs (ROC/Ps).
Over time, the assessment components designed for C-TAP evolved into an assessment system that helped instructors improve instruction in the classroom, rather than certify students for employment or further education. The program has again been adapted for another purpose. This time the focus is on assessing a smaller number of broad-based clusters of job skills and recognizing student achievement in a format similar to that of the statewide academic merit test called the Golden State Exam. According to Sri Ananda of WestEd, like the Golden State Exam, the Assessments in Career Education (ACE) is a voluntary assessment system where students can receive a seal of recognition on their diploma. In the spring of 1999, the ACE program will be administered in five career technical areas: [1] agriculture, [2] computer science and information systems, [3] health care, [4] food services and hospitality, and [5] industrial and technology education.
There have been many reasons for the continuing evolution of C-TAP including shifting political climates regarding assessment, department of education reorganization and changing purposes, the lack of centralized direction to provide incentives to participate, and consistent implementation. While the particular focus of the program has evolved over time, the primary goal of C-TAP remains the same: to provide better information to teachers, students, and other stakeholders about the achievement level of students through multiple forms of assessment. According to Daniel McLaughlin of WestEd, C-TAP is "one of the best [assessment] tools out there to teach to and assess occupational work standards." It is hoped that the information gathered through C-TAP assessment strategies will continue to help teachers improve their instruction by multiple means of measuring the broad skills and knowledge necessary to succeed in the workplace.
Implementation Strategy: The C-TAP system consists of four components: [1]multiple-choice questions, [2] written scenarios, [3] student projects, and [4] portfolios. The components are linked to Challenge Standards, formerly called Model Curriculum Standards (MCSs), in the five vocational areas and general workplace readiness standards. Because C-TAP is a voluntary system, teachers can use all of the components or choose those pieces that best suit their needs. Details of the four components follow:
- On-demand, multiple-choice/short answer tests which have been developed and are part of ACE.
- Written scenarios involving problem-solving tasks requiring immediate written responses to job-related problems. Students may "rehearse" or practice the scenario, but the official assessment is an on-demand task administered under standardized, secure testing conditions with a 45-minute time restriction. Written scenarios are scored holistically based on four levels of achievement using rubrics developed by WestEd: [1] Low Basic, [2] Basic, [3] Proficient, and [4] Advanced.
- Projects allowing students the opportunity to show their expertise and knowledge of one or two standards in their area of study. They involve a hands-on application of skills as well as an oral presentation of the knowledge and skills they learned during the course of the project. Projects include the following:
- A plan - the steps of the project and the material that will be submitted for evaluation
- Evidence of progress - three sources demonstrating the skills used and the work accomplished before submitting the final product
- Final product - which demonstrates mastery of the standards and may include the actual project or other documentation (i.e. videotape)
- Oral presentation - a description of the project, the knowledge and skills learned, and a self evaluation
- A plan - the steps of the project and the material that will be submitted for evaluation
- Portfolios that include the following:
- Presenting the Portfolio -- students introduce readers to the portfolio through a letter of introduction and a table of contents
- Career development package -- an employment or college application, a letter of recommendation, and a résumé
- Work samples -- at least four products that illustrate mastery of skills detailed in at least one MCS
- Writing sample -- a demonstration of their writing skills
- Supervised practical experience evaluation -- an illustration of skills the student has mastered in a work-based environment (This component is optional)
- Presenting the Portfolio -- students introduce readers to the portfolio through a letter of introduction and a table of contents
According to WestEd, the portfolio component has become the most widely liked and used part of C-TAP. Because of the evolving purpose and the subsequent "low-stakes" emphasis of C-TAP components to improve instruction rather than to certify students for employment, most of the focus on implementation has been to support the use of the materials in the classroom, not in investing in the technical quality of the instruments. ACE has been studied and has been shown to have adequate levels of technical quality, however. In addition, the Teacher Guidebook and Student Guidebook created by WestEd have become essential to the implementation of the program.
Evolution of Strategy: A critical barrier to statewide implementation and consistent adoption of C-TAP is due in part to shifting political winds and the reorganization of the California Department of Education. In addition, the impact of changes in the oversight of the program from Vocational Education to the Student Performance Division further affects the purpose and focus of the program. In part, C-TAP has survived these changes through teachers' perceptions of flexibility of the components and the responsiveness of the system to the needs of teachers to improve instruction and assessment together.
More recently, C-TAP has again undergone changes; this time it has gone from more strictly performance based and constructed responses to a standardized multiple-choice exam with open-ended questions. As the stakes become higher and C-TAP components are more widely used, the usability and adaptability must, in part, give way to the creation of an assessment system that is accessible statewide and administered fairly.
Outcomes/Lessons Learned: C-TAP is an example of a voluntary assessment system that has evolved to meet the needs of instructors and students in improving instruction, curriculum, and assessment. Because of its voluntary nature, there has been no consistent or standard implementation of any of the components. Teachers may select those components that best serve their needs. They use them in any order, at any time during the year, and may develop their own grading systems. While guidebooks produced by WestEd have been praised by teachers, they are designed to improve how teachers use the materials as opposed to increasing the reliability of scoring or the consistent use of the components from a teacher-to-teacher or class-to-class basis.
The benefits of C-TAP to teachers include knowledge about state standards and their integration with course objectives and a resulting modification of instruction. Writing, speaking, and self-reflection are emphasized, and vocational teachers report being encouraged to collaborate more with academic teachers. However, it is unclear how useful the C-TAP components are for students entering the workplace or applying for postsecondary training and education. In addition, as California makes plans to modify C-TAP into a student merit exam, further work will be necessary to ensure technical quality and fair implementation for all students.
Evaluation and Training Institute. (1997, October 3). Evaluation of the Career-Technical Assessment Program (C-TAP). Final report submitted to the Sacramento County Office of Education. Los Angeles: Author.
Support to States Home |