Improving Vocational Programs Through Self-Assessment, Quality Standards, and Industry Certification
Providing Regional Technical Assistance and Facilitating Self-Assessment of Programs
Each year in South Dakota, one region within the state participates in the Program Improvement Process for secondary vocational technical education. This process is "designed to help establish `where we are now, where we want to be, and how we are going to get there.'" Through completion of the "Self-Assessment of Quality Indicators" instrument, randomly selected site visits, and independent team reviews, vocational technical programs develop "Strategic Action Plans" that form the basis for further technical assistance from the state's Division of Workforce and Career Preparation (DWCP). These Strategic Action Plans address how programs plan to meet criteria and standards identified in the South Dakota State Model for vocational technical education.
Within the selected region, a random sample of programs is notified that they
will be visited by an "On-Site Planning Team" facilitated by a DWCP staff
member. The site visit selection process takes into account large, medium,
small, urban, and rural school districts. Programs not selected for a site
visit complete an Independent Team Review locally.
The first step in the process is the completion of the "Self-Assessment of Quality Indicators" instrument, developed using information from research organizations, national models, South Dakota instructors and administrators, and state-level staff. The self-assessment instrument includes criteria and standards in five broad categories: [1] curriculum and instruction planning, [2] management, [3] educational equity, [4] career guidance and counseling, and [5] vocational student organizations. The instrument also includes indicators and measures of quality vocational technical education which program instructors use as benchmarks to evaluate their own programs. Program instructors rate their own programs based on the degree to which they believe their program is addressing the criteria or standard using the following scale:
- Institutionalized: Program in place a year or more
- Functional: Program in place a year or less
- Early Implementation: Program is in the planning stages; implementation is just starting
- Planning: No implementation yet, in planning stages only
- Not Yet Considered
During the self-evaluation, program instructors also complete columns indicating a time line, the persons responsible for completing the plan, and from whom they need technical assistance to complete the plan. Technical assistance may be provided by business/industry representatives, other community resources, student services personnel, advisory committee members, and state staff. Once the self-assessment is completed, programs are instructed to prioritize their plans and goals and indicate "high priority" plans for the first year.
Programs visited by an On-Site Planning Team review the information submitted in the self-assessment and produce a draft "Strategic Action Plan," which identifies the activities necessary to complete the plan. The state facilitator then sends the draft plan to the program instructor for review and approval by the team. When the plan is approved, a copy is mailed to the school district or multidistrict administration. Programs not selected for a site visit conduct an Independent Team Review and must also submit a Strategic Action Plan to the state. All programs in the region then participate in a fall and spring inservice hosted by state staff, at which further planning and technical assistance is provided on a regional basis.
Monitoring
Programs Through State-Approved Quality Standards
As part of the state's vocational technical plan, each Area Vocational and Technical School (AVTS) in Oklahoma is accredited by the state every five years. In addition to the state accreditation process, Oklahoma has adopted a policy that programs will adhere to national industry standards where available.[3] Programs will have five years to be in compliance with national industry standards as they are adopted in state policy.
The
initial part of the state's current accreditation process consists of a
comprehensive 
self-evaluation by program instructors using the state's Summary Evaluation
form for their particular program area. While the evaluation forms for each
program area differ, all program areas follow Oklahoma's "11 standards of
quality program operations" that have been endorsed by the State Board of
Vocational and Technical Education.
After the self-evaluation, the Summary Evaluation form is then completed by an external evaluation team, consisting of the state program administrator, an industry advisor, and a teacher from another part of the state. Programs must meet all minimum standards to be in compliance. Those programs that do not are given 60 days to respond with a plan of improvement, and a time is given for adjustments to be made. At that point, programs are reviewed a second time, "but only in the areas that are substandard," according to Ivan Armstrong, state program administrator for Trade and Industry programs. Programs receive interim reviews to help them prepare for the next cycle of program evaluation.
The instruments used both for self and external evaluations are divided into eleven sections based on the standards endorsed by the State Board of Vocational and Technical Education. Under each standard, questions that reflect "quality indicators" are grouped into two categories. Minimum quality standards set by State Board Rules and Regulations
| |||||||||||||||||||||||||||||||||||
are either designated as "met" or "not met." Programs must meet all minimum standards in order to retain state certification and program funding, according to Armstrong. All other quality indicator questions are rated on a Likert scale of 1 to 5, depending on whether the program is exceeding, meeting, or falling below the standard. The table on the preceding page provides a sample minimum "quality indicator" for each of Oklahoma's 11 standards for quality program operations from the "Summary Evaluation Questionnaire for Trade and Industrial Education."
According to Armstrong, Oklahoma's program evaluation process helps to ensure that all students receive instruction in the kinds of skills that will lead to employment opportunities. Through Standard 9, the requirement that all programs develop leadership and offer students the opportunity for extensive involvement in vocational student organizations, Oklahoma students have the option of gaining the "number one thing that industry wants: a positive attitude, good work habits, general employability."
To meet the broad national perspectives of industry, beginning in 1990 the Oklahoma Department of Vocational and Technical Education established a policy that national organizations would provide program certification whenever possible. The policy established that Oklahoma programs would meet national standards and obtain certification to maintain state funding and accreditation for their programs. Examples of national certification programs in use in Oklahoma include standards from the automotive (NATEF), construction (Laborers-AGC), printing (PIA), and welding (AWS) industries. Oklahoma accepts the portions of national certification that are similar to its own standards and evaluates only those areas that are not included in the national certification program. Health programs also have specialized accrediting agencies which the State Board accepts in lieu of standards. In addition, the Oklahoma Department of Vocational and Technical Education is in the process of "crosswalking" vocational and technical competencies to state-adopted academic standards so that "students can obtain academic credit, where appropriate, for their work in vocational programs."
Requiring Business and Industry Certification Through State and National Industry Standards
Beginning
in the year 2003, all career/technical education programs in Alabama are
required to certify to industry standards. To meet this requirement, Alabama
has chosen 
five national certification programs for industry certification: [1]
automotive, [2] construction, [3] drafting, [4] metalworking, and [5] printing.[4]
All other career/technical programs will be certified according to business and
industry certification standards written at the state level. According to Nancy
Beggs, Acting Director of the Office of Career/Technical Education, Alabama's
industry standards have been written using the NATEF program standards as a
model. To assist in the effort to certify all career/technical education
programs, the state legislature passed a $20 million capital improvement bond
supporting the purchase of equipment.
In order to have all programs meet industry standards by 2003, the Alabama Office of Career/Technical Education in the Department of Education has developed procedures that each district or Local Education Agency (LEA) must follow. According to these procedures, "once a program is awarded business/industry certification, it will remain a `certified program' for five years." Following are the required procedures:
- Plan Development: Each LEA will draft a five-year Plan for Program Certification to Industry Standards so that all programs in its jurisdiction will be certified in five years. A minimum of 20% of its programs must be certified annually.
- Staff Development: All teachers must receive staff development on industry certification prior to the self-evaluation and documentation stages. Teachers may participate through a local inservice or state-sponsored staff development.
- Documentation: Documentation must be collected and clearly organized for each program standard identified in the Career/Technical Education Business Industry Certification document.
- Self-Evaluation: Through a committee of school administrators, faculty, and local advisory committee members, programs compare themselves to the certification criteria. Once completed, the program is scheduled for an on-site review.
- On-Site Review: The on-site review team consists of a career/technical education administrator/specialist from the state Department of Education who is the team leader, business and industry representatives, and outside education representatives selected by the LEA. Based on the on-site evaluation, the team will make a recommendation for industry certification. If a program is denied certification, it will be given an explanation of what must be done in order to comply.
In addition, for a program to be certified, teachers "must possess knowledge and skills as prescribed by industry standards and the department." Teachers must have a valid teaching certificate based on a bachelor's degree or higher in the teaching field or will be required to "pass an industry certification test as determined by the department." Though it is widely acknowledged that this requirement and industry certification in general will have a positive effect on career/technical education in Alabama, according to Beggs, it has caused some anxiety among teachers as the programs and their teaching qualifications come under scrutiny. To help ease this anxiety, the state has bolstered technical assistance services and has made a commitment to provide ongoing professional development in the industry skill certification process for teachers throughout the state.
Alabama Department of Education. (1998, June 12). Procedures, career/technical education business/industry certification. Montgomery: Author.
South Dakota Department of Education and Cultural Affairs, Division of Workforce and Career Preparation. (1998). Western region -- 1998-99 program improvement process: Self-assessment of quality indicators instrument. Pierre: Author.
[3] Current industry standards that are used or under consideration in Oklahoma include National Automotive Technicians Education Foundation (NATEF) (automotive), Printing Industries of America (PIA) (printing), American Welding Society (AWS) (welding), and National Institute for Metalworking Skills (NIMS) (machine tool).
[4] Industry standards in use in Alabama include NATEF (automotive), Associated General Contractors (construction), American Design Drafting Association (ADDA) (drafting), NIMS (machine tool), and PIA (printing).