Emerging Strategies for Assessment Systems
North Carolina: Vocational Competency Achievement Tracking System
North
Carolina's Vocational Competency Achievement Tracking System (VoCATS)
incorporates course competencies developed by educators and industry
representatives, and end-of-course exams to help teachers measure student
progress and adjust teaching methods, course content, and materials. Course
competencies, or Blueprints, include both academic and technical competencies
and form the basis for the curriculum and course assessments in VoCATS. The
system now includes a computerized test item
bank for virtually every course taught in the state, or about 100 of the 110
courses that are available. The assessment items are multiple-choice and are
reviewed and revised every five years. At the beginning of each year, students
take a pre-test generated by the North Carolina Department of Public
Instruction. The tests are scored locally by teachers who then scan the results
into "Testmate," a CTB-McGraw Hill management program which generates a report
detailing initial student competency. Interim tests are used throughout the
year to assess student competency and modify courses during the year. The
Department of Public Instruction administers a post-test at the end of the year
which is used both to report performance information to the state and for
instructional planning at the site. North Carolina first began to test
vocational students in the 1970s and over the years the assessment system has
continued to evolve, enabling state staff and local educators to improve the
administration and technical quality of the assessments and the vocational
education system in general.
Michigan: Employability Portfolios
Beginning
in the 1992 - 1993 school year, Michigan introduced an employability skills
portfolio for students to use in grades 8 to 12, which was intended to help
schools implement the Michigan State Board of Education's model core curriculum
which had been adopted in 1990. In particular, the portfolios addressed Career
and Employability skills, one of nine areas in the state's core curriculum. To
support more consistent implementation of the portfolio system, the Michigan
Department of Education created an Employability Skills Assessment Kit
detailing the elements of a portfolio and presented standardized procedures for
implementation at schools, workshops, and conferences. Schools with experience
in using the portfolios served as resources for other schools starting to use
the system. Although the portfolios are no longer supported as a state-level
program, portfolio materials are available, and anecdotal evidence suggests
that they continued to be used, especially in those districts and schools whose
use of the portfolios reinforced the benefits of this kind of assessment for
students. Based on employer feedback, the employability portfolios were
designed to assess skills in three areas: [1] academic, [2] personal
management, and [3] teamwork. Academic skills included communication, math,
problem solving, and science and technology. Personal management covered career
development, flexibility and initiative, organization, and responsibility.
Teamwork was also assessed based on student contributions to and membership in
a team, responsiveness, and team communication skills. Within each of the three
areas, four benchmarks and a list of skills under each area provided the basis
for assessment. Student exhibits such as pictures, videotapes, or written
documents were used to demonstrate the student's skills for each benchmark;
these were reviewed and approved by a teacher and counselor.
Ohio: Multidimensional Assessment System
In
1990, Ohio initiated a ten-year strategic plan for vocational programs. In
1996, the updated plan, "Ohio's Future at Work: Beyond 2000," continued to
reflect the state's push for increased accountability for vocational programs.
As programs strive to become more performance oriented, Ohio has found that the
availability of multiple strategies for assessment will play an increasingly
critical role in continuous improvement. Because of the expanded role of
assessment in program accountability, the division of Vocational and Adult
Education has been promoting a multidimensional
system of student assessment. Ohio currently offers Occupational Competency
Analysis Profiles (OCAPs) tests in 38 vocational program areas that measure
student competence in technical areas of instruction and Work Keys from ACT
that measure academic skills expected in the workplace. As a complement to the
state's traditional paper and pencil testing system, a variety of
performance-based assessments are being considered. Assessments structured
around real world, career-based performance tasks are currently being piloted
in business, agriculture, and the family and consumer sciences areas. Those
outside of the student's regular classroom, including other teachers, parents,
students, and business persons, are expected to play a role in the evaluation
process. While these instruments may not be used with all students, it is
anticipated that a portion of Ohio's students will have access to the
assessments through the state's assessment system. Practices that incorporate
multiple measures of student performance over time will provide the third
dimension of Ohio's assessment system. One example is a student portfolio
containing results of paper and pencil assessments, performance measures,
teacher and employer observations, and student reflections on their work. While
Ohio is not using portfolios on a statewide basis, the state is exploring how
to collect information that demonstrates student progress, perhaps through
vocational student organization activities in the assessment process.
South Carolina: Vocational Student Certification System
In
South Carolina, plans are underway to implement a state assessment system to
certify student competency in secondary vocational programs. Students who
pursue a course of study, completing four units in a vocational program area,
will be eligible to take the state assessment. Students who pass the test will
receive a certificate from the state which
can be used when they enter the workforce. If students choose to pursue further
postsecondary education, the certificate will give them documentation of the
skills they are competent in, thus giving them the opportunity to enter at a
higher level than they might otherwise. In the spring of 1999, assessments with
both performance-based and standards test elements will be piloted in 11 areas.
The assessments have been developed with the assistance of V-TECS and NOCTI and
are customized for South Carolina. Each assessment area has an advisory council
representing members of the business/industry and education communities. To
support the integration of this new assessment system in vocational courses,
South Carolina is in the process of developing crosswalks of the state's
academic standards to the program areas in the assessment and student
certification system.
Utah: Assessments for Program Accountability and Student Certification
About
five years ago, in return for increased program accountability, the Utah
legislature agreed to augment funding for vocational education. The legislature
passed a state law allowing a portion (12%) of funds for applied technology
education to be distributed as incentives to programs demonstrating student
proficiency. In order to implement this system of accountability for vocational
programs and distribute incentive funds, however, measures of student
competency had to be developed quickly. From initial tests developed in word
processing, information processing, and accounting,
Utah's student assessment system has grown to approximately 85 end-of-course
and end-of-program multiple-choice and performance assessments. Approximately
78,000 students took the exams last year, up from 4,000 in the first year. The
students' assessments are sent to the Utah Applied Technology Resource Center
for scoring. This Resource Center also provides assistance and guidelines for
teachers. A Protocol Manual has been developed as a guide to the tests and
reports sent to the vocational education program. Programs receive a portion of
their funding based on student scores. In addition, students scoring above 80 % receive a certificate that shows the competencies demonstrated by the exam.
Because the testing program had to be up and running in a relatively short
period of time, Utah is now in the process of having an outside evaluation team
examine the technical quality and reliability of its assessments. Although some
assessments may be changed as a result of the current analysis, the Utah
Department of Education has relied on the assessment system to encourage
teachers and programs to teach to uniform standards, objectives, and
competencies, thus facilitating the shift to a performance-based model in Utah.
