Using Occupational Skill Standards to Measure Student Competency
Policy Rationale and Goals: Since discussions that began in the mid-1970s, Oklahoma has been committed to developing a system of skill certification for its vocational students. Over 16 years ago, the Oklahoma Department of Vocational and Technical Education (Oklahoma Vo-Tech), a separate agency from the Oklahoma Department of Education, began to develop a competency-based testing program to measure the skills of vocational students in its comprehensive high school vocational programs and area vocational and technical centers. The Testing Division began as a segment of Evaluation and Testing Services in 1982; and in 1990, the Testing Division became a separate unit within Oklahoma Vo-Tech, growing from a staff of three in 1982 to 15 in 1998. The Testing Division's mission is to accomplish the following:
Implement a valid system to measure occupational competencies of students and industry workers, to meet the evolving certification needs of Oklahoma and select national groups:
The Testing Division works closely with vocational programs to establish occupational standards and measures for their students. To meet thecertification needs of our vocational programs and industry, the Testing Division develops and continually maintains a variety of occupational duty/task lists and tests.... Our duty/task lists are an important part of Oklahoma's system of standards and measures. They ensure that our curriculum addresses industry's needs and that vocational students have attained those skills necessary for job success. (Oklahoma Department of Vocational-Technical Education, 1998)
Implementation Strategy: The testing program as originally envisioned by business leaders and the Oklahoma Board for Vocational and Technical Education was closely linked to the framework under development by the National Occupational Information Coordinating Committee (NOICC) and the state counterpart, the Oklahoma State Occupational Information Coordinating Committee (SOICC). NOICC is a federal interagency committee, established by Congress in 1976:
NOICC has two basic missions. One is to improve communication and coordination among developers and users of occupational and career information. The other is to help states meet the occupational information needs of two major constituencies: 1) planners and managers of vocational education and job training programs and 2) individuals making career decisions. (Oklahoma Department of Vocational-Technical Education, 1992)
The NOICC and SOICC information systems are based on specific occupations. In turn, as Oklahoma built its system, it chose a testing strategy based not on broad occupational categories, industry areas, or general workplace competencies, but on specific occupations. Oklahoma's testing program builds curriculum and assessment from specific occupational duty/task lists for over 60 program areas. The validity of the duty/task lists, curriculum, and assessments is verified and endorsed by industry advisory groups for each program area.
Since its inception, the Testing Division has developed over 60 duty/task lists, addressing approximately 300 vocational occupations. According to Chuck Hopkins, an Oklahoma Vo-Tech official, starting at the occupational level is most useful for their purposes. Creating a separate duty/task list based on an occupation gives instructors information about "all the skills that are required and a foundation for the mix of tests and curriculum." In addition, "employers can identify with the system -- people are hired for specific jobs" as opposed to broad industry clusters. The duty/task lists contain employability skills that often cut across occupations, and instructors can combine information from lists to meet the needs of students in their courses.
Evolution of Strategy: Oklahoma Vo-Tech administers the program with the help of testing liaisons/assessment coordinators at the school sites. The analysis of all test results is done at the state level. According to Hopkins, the biggest changes occurring now involve "going to the Internet system" and keeping up with technology. The Testing Division envisions using the Internet for ordering curriculum and testing materials, managing the test results process, and providing information to school sites and Vo-Tech centers. Oklahoma Vo-Tech sees great promise in the use of technology for the testing program and hopes to use video-conferencing to provide inservice programs statewide. As Hopkins asserts, "Technology is the next step--as the whole system changes from teaching to learning."
Over the years, Oklahoma Vo-Tech has also developed extensive curriculum materials and information that tie closely to the duty/task lists and assessments. The Curriculum and Instructional Materials Center distributes the materials and is in the process of developing an on-line catalog for teachers and school site coordinators. The Curriculum and Testing Divisions market the materials at shows and conferences and distribute catalogs widely throughout the state and nation.
Lessons Learned: Oklahoma Vo-Tech is confident in its occupational testing program and counts among its successes the continued endorsement and validation received from industry. In addition, the state makes substantial investments in developing curriculum materials to support instructors. According to Hopkins, the Testing Division conducts regular customer surveys and focus groups and "is constantly looking at ways to improve--we go through all the hoops." They have found that while instructors are not required to use their materials, the majority consistently do.
As states implement testing systems, it is important to remember that "the perfect system is almost impossible to administer--testing great numbers of students accurately is difficult and resources do limit the scope of our dream systems," explains Hopkins. Much research is conducted, and the assessment system is tested on incumbent workers in the construction office and sales industries, measuring both "hard" and "soft" skills. The close connections maintained by Oklahoma Vo-Tech with both industry and vocational instructors have resulted in "no political opposition to the system" and it is consistently endorsed by the State Board of Vocational and Technical Education. The testing program has also maintained political support by avoiding "penalties for individuals" and "staying competency based."
Oklahoma Department of Vocational-Technical Education. (1997). NOICC information. Available on-line: <http://www.okvotech.org/testing/occptest.htm>.
Oklahoma Department of Vocational-Technical Education. (1998). Occupational testing. Available on-line: <http://www.okvotech.org/soicc/national.htm>.
