NCRVE Home | Site Search | Product Search

NCRVE Home | Full-Text Documents | Contents | Previous Section | Next Section

METHODS

Using survey data gathered during Phase One of the study, eight two-year colleges were identified for further in-depth analysis using qualitative methods. Ten work-based learning programs in eight colleges, including health and non-health programs in two of the institutions, were chosen using a multistage selection process. The following five steps summarize the process:

  1. The population of two-year college work-based learning programs for Phase Two of the study in 1994 came from a compilation of the survey data conducted in Phase One of the study in 1993. In that earlier phase, survey respondents were asked to give detailed information about their best health and non-health work-based learning programs. These self-nominations were based on respondents' assessments of existing work-based learning programs according to the following criteria:

    A total of 399 health work-based learning programs and a total of 322 non-health programs were described by the survey respondents. From this pool of nominations, the work-based learning programs were screened further based on respondents' affirmative responses to the following survey items:

  2. Using these criteria to screen the population, a subsample of approximately 70 work-based learning programs was identified. These programs were classified according to

  3. Based on this classification scheme, the programs were analyzed further to identify a mix of work-based learning programs that were also distributed by

  4. Using this classification schema and supplementary information provided by the sites, 17 colleges were identified as deserving of further consideration for the study. At this stage, all of the other detailed information collected for each program was mailed to four members of the National Council for Occupational Education (NCOE) task force on work-based learning, a key group engaged in formulating and carrying out this entire study. Each task force member was given a summary of the research objectives, the selection criteria, and classification schema and asked to rank order the sites for field visitation. Results of the expert panel showed a great deal of agreement regarding the colleges and programs that should be visited. All four of the task force members rated seven of the seventeen sites among their top ten. Three other sites were rated in the top ten by either two or three of the raters.

  5. Based on this ranking process, additional information was collected for the top ten colleges identified by the panel of experts. Telephone interviews were conducted with administrators of each of the colleges to attempt to verify information supplied previously to the research team and to clarify any questions concerning the local programs, policies, and practices. In addition, personnel from state agencies and peer institutions were interviewed to corroborate the exemplary character of the identified programs. Based on this information, four colleges were disqualified, leaving six of the ten selected by the expert panel. In addition to these six, two colleges were added to the sample to strengthen the following areas: (1) the mix of occupational-technical program areas, (2) the types of work-based learning models, and (3) the regional representation within the United States. Verification of the quality of work-based learning programs at these two colleges was conducted similarly to other sites through a review of extant survey data, analysis of supporting documentation, and telephone interviews with site administrators and peer institutions. Selection of these two additional colleges were confirmed by the NCOE expert panel.

    Based on the selection process, eight two-year colleges and ten work-based learning programs were ultimately identified for the study. The following were the selected colleges and programs:

Data Collection Procedures

The data collection was conducted by NCRVE project staff along with members of the NCOE task force on work-based learning (chaired by Russell Hamm), creating a team of ten data collectors. By combining the NCRVE researchers and NCOE practitioners into one research team, we created a working partnership that was valuable to understanding exemplary policy and practices associated with work-based learning. The partnership between the NCRVE researchers and NCOE practitioners provided a means of examining the relationship between theory and practice inherent in work-based learning itself. For each site visit, we assigned a team of two data collectors, optimally a pairing of an NCRVE researcher with an NCOE practitioner.

The Protocol

The data collection instruments and procedures used during each field visit were based on information gathered during Phase One of the study, additional information collected via a literature search, and previous field-based research conducted by the project staff, especially from prior investigations of Tech Prep programs (Bragg, 1992). A protocol was developed for the field visit and consistently employed by all the teams. Using this process met two goals: First, it ensured that a common set of questions and inquiries was applied at each site. Second, using the protocol enabled the team members to gather information consistently about organizational structures and operational strategies to identify factors that contribute to exemplary practice. Also, using the protocol enhanced the confidence among team members in their ability to conduct the research study and contribute to the data collection in a quality manner.

The protocol was based on the research objectives for the study, and it contained a core set of research questions that provided the foundation for all data collection activities (see Table 2). Once the core set of research questions was developed, a list of potential interview questions was generated for each specific stakeholder group associated with the work-based learning programs. The questions were combined into a semistructured interview guide for each stakeholder group. These groups were as follows:

Table 2
Research Questions

What are the characteristics and mission of the two-year college?

What are the characteristics and goals of the program?

Who are the stakeholders: Students, Faculty, Employers, Others?

What are the components of the program?

What are the strengths and weaknesses (obstacles) of the program?

What are the outcomes/results of the work-based learning program?

What lessons have been learned about work-based learning?

Pilot Test

A pilot test was conducted in July 1995 with two programs at Columbus State Community College (CSCC) in Columbus, Ohio. With preliminary methodology and instrumentation developed, four members of the data collection team made a visit to CSCC to investigate two programs: culinary arts and nursing technology. Two full days were spent on site to collect data using the protocol which was based largely on qualitative research methods described by Patton (1980). Information gathered at the visit was based on the protocol and research questions presented in Table 2.

Following this pilot test, project staff made final revisions to the data collection protocol and the semistructured interview schedules. Also at this time, brief paper-pencil surveys were created to collect data from persons difficult to reach in sufficient number during the visit, especially students, faculty, and workplace mentors. These surveys were intended for all faculty and workplace mentors actively engaged in the program. In addition, the research teams were given thirty student surveys and asked to collaborate with the personnel at each site to collect information from a representative group of students. (Sampling procedures varied because the programs varied widely in student enrollments from only 10 to nearly 340.) The student surveys collected demographic information and perceptions toward various aspects of work-based learning.

In addition, a school-to-work assessment tool was created to provide a measure of program quality independent from the observations made by local stakeholders. The instrument provided a scaled response to identify the extent to which various aspects of the STWO Act were implemented. In addition, the instrument included a scale to indicate the level of quality of various elements associated with the three components of the STWO Act: (1) school-based learning, (2) work-based learning, and (3) connecting activities. The instrument was completed independently by each research team member and then the teams convened to discuss their observations. The teams were asked to complete the assessment tool as a group, reflecting the consensus of all team member responses. Both the individual team member responses and the group consensus were contributed to the pool of data collected for this study.

Training of Data Collectors

Once the data collection instruments and procedures were finalized, the team of ten researchers met for a training session. The training was conducted over a two-day period in late August 1994 in Chicago, Illinois. It provided the following:

The Field Visits

During September and October of 1994, the NCRVE-NCOE research teams conducted site visits to seven two-year colleges. Each visit was conducted over two days or more, typically involving 10- to 12-hour days of on-site interviews and observations. On each site visit, the two-year college and at least two employers where students were actively engaged in work-based learning were visited. In addition, other organizations critical to operation of the program(s) were sometimes visited such as a local Chamber of Commerce or labor organization.

In each visit, persons affiliated with the key stakeholder groups were interviewed. Personal interviews were conducted individually or in small groups, depending upon the persons being interviewed, the confidentiality of information sought, and the accessibility of the interviewees. Data collected via the interviews were recorded in written notes. In nearly all cases, cassette tape recordings were used as an additional source of documentation. Survey instruments, along with pre-addressed and stamped envelopes, were distributed to students, faculty, and mentors to supplement the on-site data collection. Based on a compilation of all of this data, each team was responsible for drafting a case study report to document what was learned in relation to the research objectives. The format for the case-study reports paralleled the research protocol used in the data collection activities. The following are the primary sections of the case studies:

Data Analysis and Reporting

The process of analyzing and reporting data for this report was a bit unconventional because the data collected via the eight field visits resided primarily in the heads of the ten members of the research team. Although we did require extensive note taking, and tape recording when possible, much of the understanding and interpretation resided within the people themselves. Therefore, to aggregate the major findings across the sites; to identify common patterns, themes, and issues; and to generate conclusions and recommendations, the ten-member research team was convened for a debriefing session. Draft copies of each case study report were prepared and shared with the entire team for assistance in the debriefing process. Later, after the session, each team finalized its case study report, providing the basis for this document.

Debriefing Session

After the data was collected in late October 1994, all ten data collectors gathered for a one-day meeting held during the fall conference of NCOE in Chicago, Illinois. This session was designed to assist in finishing the individual case study reports as well as to provide a forum for discussion of the major findings, conclusions, and recommendations of the study. The group discussion was facilitated by the project co-directors using a protocol developed prior to the session. The protocol involved having each research team provide a brief description of major findings, lasting no more than 30 minutes per presentation.

Following the overview given for each program, the discussion focused on identifying factors thought to be associated with exemplary work-based learning policy and practice. These factors were defined with examples shared by individual researchers, and then the group was polled to determine the incidence of the factors within particular programs. When the factors seemed to be representative of all or nearly all of the sites, the factors were defined in a way that was broadly representative of what was observed across sites. Ideas that seemed more idiosyncratic were not discarded, but recorded to assist in further interpretation of the data or in generating future research and policy questions. At the completion of the day-long meeting, the research team had generated an extensive amount of information about two-year college work-based learning policies and practices useful in addressing the study's research objectives. The final part of the discussion centered on issues and concerns evident in the work-based learning programs studied. All ideas generated by the research team were noted on flip charts to assist in data analysis during the meeting and during later report writing. The entire session was video and audio taped to assist in documenting the event. The authors of this report took primary responsibility for synthesizing and reporting the ideas generated during the debriefing session.

Preparation of this Report

Following the debriefing session, each team of data collectors finalized its case study report and transmitted the report and all information pertinent to the research to the NCRVE-University of Illinois site. Then, all case studies were organized into a common format and edited. Draft versions of each case were mailed to the field-study sites for verification of the accuracy of information. In addition, notes and audiotapes obtained during the debriefing session in Chicago were transcribed, reviewed, and used as a basis for the findings and conclusions presented in this report. A draft of the final report was shared with each member of the research team for review and critique.


NCRVE Home | Full-Text Documents | Contents | Previous Section | Next Section
NCRVE Home | Site Search | Product Search