Using Database Methods To Improve Accountability and Results
Policy Rationale and Goals:
In 1993, the Illinois State Board of Education (ISBE) responded to the Carl D.
Perkins Act of 1990 requirement that states create performance measures for
local vocational education programs by creating the Illinois Performance
Management Information System (IPMIS). Because ISBE realized that creating a
data system would require the buy-in of staff at the local level, IPMIS was
designed with feedback from local administrators, educators, student services
personnel, and parents. In addition, ISBE was careful to make the process of
using 
data easier rather than creating more reporting tasks for already-burdened
administrators. IPMIS uses existing secondary, postsecondary, and employment
databases; sets performance measurement goals based on a model of continuous
improvement; and avoids ranking individual programs against one another. IPMIS
is also designed to be "user-friendly" and applies the latest computer
technology to enhance the system.
For the state, IPMIS functions as both an information and an accountability system, counting student enrollment for reimbursement purposes and allowing the state to track programs; it is not intended to be punitive, however. According to Mary Ann Merano of ISBE, the principal consultant for IPMIS, rather than compare programs and schools against each other, IPMIS data compares programs to standards that are defined with the help of local educators, administrators, student service staff members, and parents. If a particular program's scores are not meeting state standards for two years in a row, the state steps in with technical assistance: "The IPMIS represents an `agreement' between local educational agencies and the state office. It ... is one of many resources that should be used for planning, monitoring, evaluating and restructuring programs."
Implementation Strategy: Each school year, ISBE personnel in the management information systems office "cull a big file that contains records for grades 9-12 and for community college students or completers." (Miguel and Marano, 1998, p.22) They also separate data for certain student cohorts that the state wants to evaluate, such as graduated seniors matched to a list of freshmen enrolled in the state's postsecondary institutions, military enrollment records, economically disadvantaged students, or those with limited English proficiency.
After preparing the data at the state level, regional directors in the state's 60 Education for Employment offices and its 40 community colleges receive diskettes containing three years' worth of data for their district, statewide comparative summary data, and the software to access it. Regional offices have the option of running reports for the schools in their district or distributing the software to the schools.
IPMIS was purposely designed to be as user-friendly as possible so that all potential users would feel comfortable with the new program quickly. Numerous reports and graphs can be created for each individual site with only a few clicks of a mouse, providing tailored information to administrators and instructors almost instantly. The reports that are generated through IPMIS allow local practitioners to see what is working and what is not.
Evolution of Strategy: The success of IPMIS has been acknowledged both by those at the local and state levels. Teachers and administrators find the reports to be easy to use and conducive to planning strategies for program improvement. By using data to locate students after high school and at the postsecondary level, teachers and administrators have the opportunity to analyze what they need to do. Some results include sharper attention to career planning efforts, more focused staff development, and improved collaboration among vocational and academic instructors.
For state-level administrators, other facts speak volumes about the success of IPMIS. Although it is a voluntary system tied to performance measures, people at the local level are using it. In fact, local-level administrators and instructors are devising new ways to link existing data -- such as reviewing secondary and postsecondary information at the same time -- to better track where students are going after high school. In addition, IPMIS is a flexible data management system that can easily be modified and will grow according to the state's needs. For instance, the state plans to use IPMIS to better coordinate services with other state agencies and for use in regional planning.
Another clear sign of success is that the state is considering modifying how program performance is defined as a result of the data provided by IPMIS. According to Richard Miguel of ISBE's Office of Workforce Preparation, because IPMIS data monitors and tracks achievement and improvement over time, Illinois is considering attaching additional accountability components to the system. Legislation is pending in the state to provide performance incentive funds for those programs that are improving. Clear performance measures have been tied to IPMIS and are being phased in over time; and ISBE plans to use these to determine the allocation of proposed performance incentive funds.
Outcomes/Lessons Learned: Not surprisingly, one of the most important lessons learned through this process is one about data. Illinois found that while it is often difficult to collect accurate data, it is an even bigger challenge to get people to use data. According to Miguel, everyone's comfort level regarding how to use data was even lower than anticipated. In order to tackle this issue head on, Illinois has invited NCRVE into the state to conduct three training sessions for local users of data in an effort to expand their technical knowledge. Illinois has found that post-program data is particularly useful at the local level, saving programs the time and resources needed to collect this kind of data.
Illinois also learned that the flexibility and adaptability of IPMIS was the key to opening a multitude of possibilities for its application, allowing the system to be useful to more people in more ways than would have been possible if it had been designed for one program or focus. By giving state officials, local administrators, and instructors the opportunity to use data to improve their own capacity, Illinois is finding that everyone is better able to meet the needs of students.
Miguel, R. and M. Marano. Windows on Progress. Techniques (1998, March, p. 22-23).
