TECHNICAL APPENDIX

1. reappointing a State Committee of Practitioners (Section 115);
2. conducting a State needs assessment in the first year under reauthorization and every two years thereafter (Section 116);
3. implementing a set of core performance measures and standards (Section 117);
4. conducting annual evaluations of program effectiveness (Section 118); and
5. providing technical assistance to local recipients on using performance data to improve vocational courses and programs (Section 119).

New Section 115(a) includes as components of an overall improvement system the Committee of Practitioners, State needs assessment, core performance measures and standards, and annual program evaluations, all of which were introduced in the 1990 Act. Additionally, the 1995 Act adds an emphasis on providing technical assistance to local recipients as a key component of the improvement system.

The concepts that were covered in Section 115 of the 1990 Perkins Act-the Committee of Practitioners and the core performance measures and standards-are included in the proposed revised legislation under Sections 115(b) and 117.

1. Outcome-a measurable characteristic of student or program performance;
2. Measure-a method for quantifying performance in an outcome area;
3. Standard-the desired level of attainment on a measure, or the rate of progress on a measure;
4. Program-a sequence of courses or instruction in a sequence or aggregation of occupational competencies that includes occupationally specific skills, general employability skills, work attitudes, applied academic learning, higher-order reasoning, and problem-solving skills;
5. Special populations-includes individuals with disabilities, educationally and economically disadvantaged individuals (including foster children), individuals of limited English proficiency, individuals in programs nontraditional for their sex, and individuals in correctional institutions.

The State Board shall also define the term "substantial progress" in its State plan so that the term is measurable and applies consistently to all recipients of Perkins funds in the state.

1. Information on outcomes
   1. student gains in and/or attainment of academic, occupational, and/or generic work skills;
   2. student retention in and/or completion of secondary or postsecondary education and/or vocational programs;
   3. student placement into additional training or education, military service, and employment;
   4. other outcome areas identified by the State under Section 115 of the 1990 Perkins Act;
2. Information on instructional practices and experiences-
   1. integration of academic and vocational education;
   2. the ability of the eligible recipients to meet the needs of special populations with respect to vocational education, including providing incentives for participation, retention, and completion;
   3. assessing the quality of vocational education programs in schools with high concentrations of poor or low-achieving students;
   4. enrollment, retention, and completion rates of students in vocational programs nontraditional for their sex;
   5. sequential courses of study leading to both academic and occupational competencies;
   6. linkages between secondary and postsecondary educational institutions;
   7. instruction and experience, to the extent practicable, in all aspects of the industry the students are preparing to enter;
   8. the relevance of programs to the workplace and to the occupations for which students are to be trained, and the extent to which such programs reflect a realistic assessment of current and future labor market needs, including needs in areas of emerging technologies;
   9. the ability of the vocational curriculum, equipment, and instructional materials to meet the demands of the workplace;
   10. basic and higher order current and future workplace competencies that will reflect the hiring needs of employers;
   11. other factors considered appropriate by the State Board; and
3. Procedures for using existing resources and methods developed in other programs receiving Federal assistance.

(3) increased student work skill attainment and job placement;

(1) integration of academic and vocational education;

(6) the ability of the eligible recipients to meet the needs of special populations with respect to vocational education;

(7) raising the quality of vocational education programs in schools with high concentrations of poor and low-achieving students;

(2) sequential course of study leading to both academic and occupational competencies;

(4) increased linkages between secondary and postsecondary educational institutions;

(5) instruction and experience, to the extent practicable, in all aspects of the industry the students are preparing to enter;

(8) the relevance of programs to the workplace and to the occupations for which students are to be trained, and the extent to which such programs reflect a realistic assessment of current and future labor market needs, including needs in areas of emerging technologies;

(9) the ability of the vocational curriculum, equipment, and instructional materials to meet the demands of the workplace;

(10) basic and higher order current and future workplace competencies which will reflect the hiring needs of employers; and

(11) other factors considered appropriate by the State Board.

In particular, new Section 116(a)(1) directs the State Board to examine outcome data generated through the performance measures and standards system developed under the 1990 Perkins Act in conducting its needs assessment.

Section 116(a)(2) requires the State Board to examine the instructional practices and experiences that contribute to creating quality vocational education programs. Subsequently, in new Section 118(a)(3), local recipients are required to consider the contribution of these instructional practices and experiences to their performance on the core measures and standards.

New Section 116(a)(2)(D) reflects the sex equity concerns in the 1990 Perkins Act. These concerns are again echoed in new Section 117(b)(5) as part of the core performance measures and standards.

New Section 116(a)(3) was contained in Section 115(b)(4) of the 1990 Perkins Act as part of the system of performance measures and standards. We believe it is more appropriate to examine these procedures as part of the State needs assessment.

SEC. 117. STATE AND LOCAL MEASURES AND STANDARDS
(a) GENERAL AUTHORITY-Each State Board receiving funds under this Act shall revise and implement a set of core standards and measures of performance for secondary and postsecondary vocational education programs. To advise the State Board, the Committee shall-

1. evaluate the reliability, validity, and lack of bias of the measures and standards developed under Section 115 of the 1990 Perkins Act;
2. assess the extent to which data generated under Section 115 of the 1990 Perkins Act were used effectively by recipients to improve programs and identify barriers to effective use;
3. recommend revisions to existing measures and standards; and
4. develop new measures and standards as required under Subsection (b).

Revisions to the core measures and standards will be recommended by the Committee and approved by each State Board no later than one year after the enactment of the 1995 Perkins Act. This core shall apply to all programs (as defined in Section 115(c)(4)) receiving funds under Title II of this Act. Additionally, the State Board may require that the core apply to any programs funded under Title III such as consumer and homemaking education and tech-prep. Eligible recipients may make local modifications to such core based on economic, geographic, or demographic factors, or the characteristics of the population to be served. Such modifications shall conform to the assessment criteria contained in the State plan. The State Board shall convene the Committee on a regular basis to review, comment on, and propose revisions to the State's core standards and measures of performance for vocational programs.

SEC. 115. (a) GENERAL AUTHORITY- Each State Board receiving funds under this Act shall develop and implement a statewide system of core standards and measures of performance for secondary and post-secondary vocational education programs.

. . . such system shall be developed and implemented before the end of the 2-year period beginning on the date of the enactment of the Carl D. Perkins Vocational and Applied Technology Education Act Amendments of 1990 and shall apply to all programs assisted under this Act. Eligible recipients may make local modifications to such system based on economic, geographic, or demographic factors, or the characteristics of the population to be served. Such modifications shall conform to the assessment criteria contained in the State plan. The State Board shall convene the Committee on a regular basis to review, comment on, and propose revisions to a draft State proposal, which the State Board shall develop, for a system of core standards and measures of performance for vocational programs.

New Section 117(a) directs the State Board to revise the core measures and standards developed under the 1990 Perkins Act with regard to technical quality and the effectiveness of their past use for improving programs. The new section also eliminates use of the term "system" when referring to the core measures and standards, reserving that term instead for the overall "system" for improving vocational education as described in new Section 115(a).

Under new Section 117(a) the State Board is encouraged to extend the core performance measures and standards to cover Title III programs such as consumer and homemaking education and Tech Prep.

The option to make local modifications to the system has been maintained.

1. a measure of student learning gains in the achievement of academic skills;
2. 1 or more of the following measures of work-related skills-
   1. occupational competency attainment in the industry a student is preparing to enter;
   2. attainment of work readiness or generic work skills;
   3. attainment of understanding of all aspects of the industry a student is preparing to enter;
3. 1 or both of the following measures of retention or completion-
   1. retention in or completion of secondary or postsecondary education;
   2. retention in or completion of secondary or postsecondary occupational programs or other programs funded under Title III, as appropriate;
4. a measure of placement into additional training or education (including placement into the postsecondary component of tech-prep or other School-to-Work transition program), military service, or employment; and
5. a measure of access and equity such as rates of enrollment in, retention in, and/or completion of occupational or other programs for special populations in comparison to non-special populations, including male and female students in programs nontraditional for their sex.

1. measures of learning and competency gains, including student progress in the achievement of basic and more advanced academic skills;
2. 1 or more measures of performance, which shall include only-
   1. competency attainment;
   2. job or work skill attainment or enhancement including student progress in achieving occupational skills necessary to obtain employment in the field for which the student has been prepared, including occupational skills in the industry the student is preparing to enter;
   3. retention in school or completion of secondary school or its equivalent; and
   4. placement into additional training or education, military service, or employment;
3. incentives or adjustments that are-
   1. designed to encourage service to targeted groups or special populations; and
   2. for each student, consistent with the student's individualized education program developed under Section 614(a)(5) of the Education of the Handicapped Act, where appropriate; and
4. procedures for using existing resources and methods developed in other programs receiving Federal assistance.

1. consider standards and measures developed under job opportunities and basic skills training programs established and operated under a plan approved by the Secretary of Health and Human Services that meets the requirements of Section 402(a)(19) of the Social Security Act;
2. consider standards prescribed by the Secretary of Labor under Section 106 of the Job Training Partnership Act;
3. coordinate with State efforts to implement performance measures developed under Title IV, Section 402 of the School-to-Work Opportunities Act;
4. build on the content and student performance standards and related assessments developed under Title II, Part B of the Goals 2000: Educate America Act in assessing student learning gains in the achievement of academic skills at the secondary level; and
5. build on the industry skill standards and related assessments and certifications developed under Title V of the Goals 2000: Educate America Act (the National Skill Standards Act) in assessing student attainment of occupational competencies at the secondary level, and at the postsecondary level as appropriate.

1. standards and measures developed under job opportunities and basic skills training programs established and operated under a plan approved by the Secretary of Health and Human Services that meets the requirements of Section 402(a)(19) of the Social Security Act; and
2. standards prescribed by the Secretary of Labor under Section 106 of the Job Training Partnership Act.

1. The Committee shall make recommendations to the State Board with respect to modifying standards and measures to be used under this section, based on the information provided under paragraph (2) and on the state needs assessment described under Section 118.
2. In order to improve the system of measures and standards as directed under Subsection (a), the State Board shall provide to the Committee information concerning differing types of standards and measures, including-
   1. the advantages and disadvantages of each type of standard or measure;
   2. instances in which such standards and measures have been effective;
   3. instances in which such standards and measures have not been effective;
   4. information about standards, measures, and assessment instruments already in use statewide that may be incorporated into the core set of performance measures and standards; and
   5. information provided through federal technical assistance as described under Section 119, as appropriate.
3. In the event that the State Board does not accept the Committee's recommendations made as required by paragraph (1), the State Board shall set forth in the State plan its reasons for not accepting such recommendations.

1. The Committee shall make recommendations to the State Board with respect to modifying standards and measures to be used under this section, based on the information provided under paragraph (2).
2. To assist the Committee in formulating recommendations under paragraph (1), the State Board shall provide to the Committee information concerning differing types of standards and measures, including-
   1. the advantages and disadvantages of each type of standard or measure;
   2. instances in which such standards and measures have been effective; and
   3. instances in which such standards and measures have not been effective.
3. In the event that the State Board does not accept the Committee's recommendations made as required by paragraph (1), the State Board shall set forth in the State plan its reasons for not accepting such recommendations.

New Section 117(e) requires the State Board to submit to the Department of Education as part of its State plan a description of its core measures and standards as well as a plan for providing technical assistance to local recipients. This provision emphasizes the importance of technical assistance for ensuring that performance data will be used effectively by local recipients to improve programs.

1. evaluate the effectiveness of programs with respect to performance on each measure and standard developed under Section 117;
2. evaluate the performance of special populations in comparison to non-special populations on each such measure and standard;
3. evaluate the contribution of the instructional practices and experiences identified under Section 116(a)(2) to performance on the measures and standards;
4. evaluate the contribution of any projects, services, and activities conducted with assistance under this Act to performance on the measures and standards; and
5. identify and adopt strategies to improve performance on the measures and standards.

1. review programs, with the full and informed participation of representatives of individuals who are members of special populations, to-
   1. identify and adopt strategies to overcome any barriers which are resulting in lower rates of access to vocational education programs or success in such programs for individuals who are members of special populations; and
   2. evaluate the progress of individuals who are members of special populations in vocational education programs assisted under this Act; and
2. evaluate the progress of vocational education programs assisted under this Act in providing vocational education students with strong experience in and understanding of all aspects of the industry the students are preparing to enter.

1. a description of vocational education and career development strategies designed to achieve progress in improving the effectiveness of the program conducted with assistance under this Act; and
2. if necessary, a description of strategies designed to improve supplementary services provided to individuals who are members of special populations.

1. a description of the technical assistance and program activities the State will provide to enhance the performance of the recipient;
2. a reasonable timetable to improve recipient performance under the plan;
3. a description of vocational education strategies the recipient will adopt to improve performance on the measures; and
4. if necessary, a description of strategies the recipient will adopt to improve supplementary services provided to individuals who are members of special populations.

1. a description of the technical assistance and program activities the State will provide to enhance the performance of the eligible recipient;
2. a reasonable timetable to improve the school performance under the plan;
3. a description of vocational education strategies designed to improve the performance of the program as measured by the evaluation; and
4. if necessary, a description of strategies designed to improve supplementary services provided to individuals who are members of special populations.

SEC. 119. TECHNICAL ASSISTANCE (a) FEDERAL ASSISTANCE- The Secretary shall provide technical assistance to the States, directly or by grant, contract, or cooperative agreement, with respect to the development and implementation of systems under Section 115 and subsequent sections, including assistance on-

1. setting performance standards;
2. evaluating the reliability, validity, and lack of bias of measures and standards;
3. selecting and developing appropriate assessment tools, including methods for assessing academic gains at the postsecondary level;
4. coordinating Perkins measures and standards with other evolving and emerging federal initiatives such as National Industry Skill Standards and School-to-Work performance measures;
5. collecting and disseminating information on models for using performance data for program improvement, particularly at the local level; and
6. planning for and providing technical assistance to local recipients.

SEC. 115. (e) TECHNICAL ASSISTANCE- The Secretary shall provide technical assistance to the States with respect to the development of systems under Subsection (a). In providing such assistance, the Secretary shall utilize existing resources in other Federal agencies.

New Section 119 substantially expands the technical assistance requirements at both the state and federal levels. These requirements address many of the shortcomings in State implementation of Perkins performance measures and standards found by Stecher et al. (1994) in their interim report. Specifically, under new Section 119(a), the Secretary must assist the States with the technical quality of their measures and standards, the lack of appropriate assessment tools, coordination with other workforce preparedness initiatives, and local implementation of a program improvement process.

1. assistance on evaluating the effectiveness of programs based on the performance measures and standards developed under Section 117;
2. assistance on identifying and adopting appropriate strategies for improving performance on such measures and standards;
3. training local administrators and instructors in the use of performance data to improve vocational courses and programs based on the assessment conducted under Section 117(a); and
4. disseminating to local recipients examples of the effective use of performance data to improve vocational courses and programs.

1. a detailed description of the status of each State's system for improving vocational education at the State and local level developed as required by Section 115 and subsequent sections;
2. an assessment of the reliability, validity, and lack of bias of the measures and standards adopted under Section 117; and
3. an assessment of the degree to which the use of these systems have led to program improvement at the State and local levels.

1. a detailed description of the status of each State's system of standards and measures developed as required by this section;
2. an assessment of the validity, predictiveness, and reliability of such standards and measures, unbiased to special populations, in the areas of academic achievement, vocational skill competencies, employment outcomes, and postsecondary continuation and attainment; and
3. an evaluation of the comparability of State-developed performance standards across States to establish a core of common indicators.