METHODS

1. The population of two-year college work-based learning programs for Phase Two of the study in 1994 came from a compilation of the survey data conducted in Phase One of the study in 1993. In that earlier phase, survey respondents were asked to give detailed information about their best health and non-health work-based learning programs. These self-nominations were based on respondents' assessments of existing work-based learning programs according to the following criteria:
   • the existence of a formal structure for work-based learning
     
   • commitment to work-based learning by various stakeholder groups
     
   • a proven track record of successes
     
   • innovative teaching and learning practices
     

   A total of 399 health work-based learning programs and a total of 322 non-health programs were described by the survey respondents. From this pool of nominations, the work-based learning programs were screened further based on respondents' affirmative responses to the following survey items:
   • coordinated classroom and workplace learning
     
   • integrated occupational-technical and academic instruction
     
   • formal contracts or cooperative agreements with institutional partners
     
   • formal governing/advisory board composed of institutional partners
     
   • recognized credentials of occupational and academic mastery of completers
     

2. Using these criteria to screen the population, a subsample of approximately 70 work-based learning programs was identified. These programs were classified according to
   • whether the program was health or non-health.
     
   • the work-based learning model being employed (e.g., clinical-professional, co-op).
     
   • the year the program was established.

3. Based on this classification scheme, the programs were analyzed further to identify a mix of work-based learning programs that were also distributed by
   • regions of the United States (e.g., Midwest, Southwest, Northwest)--to address regional differences in postsecondary systems, labor markets, and so forth.
     
   • rural (or small town) and urban sites--to address concerns about work-based learning in areas thought to be the most difficult to reach according to the STWO Act.
     
   • occupational-technical program areas (e.g., nursing, radiologic technology, management, early childhood)--a priority was given to non-health areas because of the lack of information about these programs, although a commitment was made to select a few health programs for comparison.

4. Using this classification schema and supplementary information provided by the sites, 17 colleges were identified as deserving of further consideration for the study. At this stage, all of the other detailed information collected for each program was mailed to four members of the National Council for Occupational Education (NCOE) task force on work-based learning, a key group engaged in formulating and carrying out this entire study. Each task force member was given a summary of the research objectives, the selection criteria, and classification schema and asked to rank order the sites for field visitation. Results of the expert panel showed a great deal of agreement regarding the colleges and programs that should be visited. All four of the task force members rated seven of the seventeen sites among their top ten. Three other sites were rated in the top ten by either two or three of the raters.

5. Based on this ranking process, additional information was collected for the top ten colleges identified by the panel of experts. Telephone interviews were conducted with administrators of each of the colleges to attempt to verify information supplied previously to the research team and to clarify any questions concerning the local programs, policies, and practices. In addition, personnel from state agencies and peer institutions were interviewed to corroborate the exemplary character of the identified programs. Based on this information, four colleges were disqualified, leaving six of the ten selected by the expert panel. In addition to these six, two colleges were added to the sample to strengthen the following areas: (1) the mix of occupational-technical program areas, (2) the types of work-based learning models, and (3) the regional representation within the United States. Verification of the quality of work-based learning programs at these two colleges was conducted similarly to other sites through a review of extant survey data, analysis of supporting documentation, and telephone interviews with site administrators and peer institutions. Selection of these two additional colleges were confirmed by the NCOE expert panel.

   Based on the selection process, eight two-year colleges and ten work-based learning programs were ultimately identified for the study. The following were the selected colleges and programs:

   • Rowan-Cabarrus Community College, Salisbury, NC - early childhood education, using the clinical-professional and school-based enterprise models
     
   • Wenatchee Valley Community College, Wenatchee, WA - tree fruit production, using both the co-op and school-based enterprise models, along with Tech Prep
     
   • Phoenix College, Phoenix AZ - management/marketing, using the co-op model
     
   • Northwestern Michigan College, Traverse City, MI - resort management, using the co-op and school-based enterprise models
     
   • Delgado Community College, New Orleans, LA - culinary arts, using formal apprenticeships, and radiologic technology, using the clinical-professional model
     
   • Columbus State Community College, Columbus, OH - culinary arts, using formal apprenticeships, and nursing technology, using the clinical-professional model
     
   • Rock Valley College, Rockford, IL - manufacturing technology, using the youth apprenticeship model, along with Tech Prep
     
   • Tulsa Junior College, Tulsa, OK - manufacturing technology, using the youth apprenticeship model
     

Data Collection Procedures

The Protocol

The data collection instruments and procedures used during each field visit were based on information gathered during Phase One of the study, additional information collected via a literature search, and previous field-based research conducted by the project staff, especially from prior investigations of Tech Prep programs (Bragg, 1992). A protocol was developed for the field visit and consistently employed by all the teams. Using this process met two goals: First, it ensured that a common set of questions and inquiries was applied at each site. Second, using the protocol enabled the team members to gather information consistently about organizational structures and operational strategies to identify factors that contribute to exemplary practice. Also, using the protocol enhanced the confidence among team members in their ability to conduct the research study and contribute to the data collection in a quality manner.

The protocol was based on the research objectives for the study, and it contained a core set of research questions that provided the foundation for all data collection activities (see Table 2). Once the core set of research questions was developed, a list of potential interview questions was generated for each specific stakeholder group associated with the work-based learning programs. The questions were combined into a semistructured interview guide for each stakeholder group. These groups were as follows:

• President and senior administrators of the college
  
• Program director(s)
  
• Faculty (occupational and academic)
  
• Counselors and support staff
  
• Students
  
• Employers (minimum of two) and workplace mentors
  
• Other groups (e.g., labor organizations, Chamber of Commerce)
  

Table 2
Research Questions

What are the characteristics and mission of the two-year college?

• Institutional characteristics, outcomes
• Student enrollments--demographics
• Funding levels and sources
• Faculty/mentor characteristics--demographics

What are the characteristics and goals of the program?

• What are the specific goals of the program?
• What was the motivation for starting the program?
• How does the program fit with the mission of the college? Employers' missions?

Who are the stakeholders: Students, Faculty, Employers, Others?

• Stakeholder demographics (e.g., age, ethnicity/race, gender)
• For each, what is the role and experience with the program?
• What was the motivation for getting involved with the program?
• What resources did each group contribute and receive?

What are the components of the program?

• What model (e.g., co-op, youth apprenticeship) is being implemented? Why?
• What are the key components of the model?
• How does the model work?
• Who has responsibility for the various components of the model?
• What is the past experience with this model?
• How do the key components fit the local area (context)?
• What is particularly innovative about the model?
• How do the following components fit?
  • School-based learning components
  • Work-based learning components
  • Connecting activities

What are the strengths and weaknesses (obstacles) of the program?

• What obstacles have been overcome? How?
• What obstacles remain? How do you plan to approach them?
• What are the obstacles to doing more work-based learning?

What are the outcomes/results of the work-based learning program?

• What are the documented student outcomes of this program?
  • Retention rates
  • Completion rates
  • Transfer rates
  • Placement rates
  • Licensure passage rates
  • Earnings
  • Job/career positions

• How are outcomes (benefits) evaluated? How are negative outcomes handled?
• How is outstanding performance/participation recognized (formally or informally)?
• What are the other outcomes of the program? The college? Employers? Others?
  
  • Public relations
  • Relationships with board/community
  • Changes in enrollment in other programs
  • Economic development
  • Human resource development

What lessons have been learned about work-based learning?

• If you could do it all over again, what would you do?
• What advice would you give to others just beginning a work-based learning program like this one?

Pilot Test

A pilot test was conducted in July 1995 with two programs at Columbus State Community College (CSCC) in Columbus, Ohio. With preliminary methodology and instrumentation developed, four members of the data collection team made a visit to CSCC to investigate two programs: culinary arts and nursing technology. Two full days were spent on site to collect data using the protocol which was based largely on qualitative research methods described by Patton (1980). Information gathered at the visit was based on the protocol and research questions presented in Table 2.

Following this pilot test, project staff made final revisions to the data collection protocol and the semistructured interview schedules. Also at this time, brief paper-pencil surveys were created to collect data from persons difficult to reach in sufficient number during the visit, especially students, faculty, and workplace mentors. These surveys were intended for all faculty and workplace mentors actively engaged in the program. In addition, the research teams were given thirty student surveys and asked to collaborate with the personnel at each site to collect information from a representative group of students. (Sampling procedures varied because the programs varied widely in student enrollments from only 10 to nearly 340.) The student surveys collected demographic information and perceptions toward various aspects of work-based learning.

In addition, a school-to-work assessment tool was created to provide a measure of program quality independent from the observations made by local stakeholders. The instrument provided a scaled response to identify the extent to which various aspects of the STWO Act were implemented. In addition, the instrument included a scale to indicate the level of quality of various elements associated with the three components of the STWO Act: (1) school-based learning, (2) work-based learning, and (3) connecting activities. The instrument was completed independently by each research team member and then the teams convened to discuss their observations. The teams were asked to complete the assessment tool as a group, reflecting the consensus of all team member responses. Both the individual team member responses and the group consensus were contributed to the pool of data collected for this study.

Training of Data Collectors

Once the data collection instruments and procedures were finalized, the team of ten researchers met for a training session. The training was conducted over a two-day period in late August 1994 in Chicago, Illinois. It provided the following:

• an orientation to the goals of the study

• a means of sharing major findings, conclusions, and recommendations from Phase One of the study

• clarification about policies, terminology, and practices considered integral to work-based learning for this study

• a general understanding of the STWO Act and the importance of examining elements of the legislation in relation to this investigation

• a discussion of the issues pertinent to existing work-based learning programs, particularly those linked to various models (e.g., youth apprenticeship, co-op) and urban and rural locations

• an in-depth understanding of the site visit procedures and data collection protocol

• instructions and requirements for creating a case study report for each site visit

• the schedule of activities for the remainder of the project, including the site visits and debriefing session

The Field Visits

During September and October of 1994, the NCRVE-NCOE research teams conducted site visits to seven two-year colleges. Each visit was conducted over two days or more, typically involving 10- to 12-hour days of on-site interviews and observations. On each site visit, the two-year college and at least two employers where students were actively engaged in work-based learning were visited. In addition, other organizations critical to operation of the program(s) were sometimes visited such as a local Chamber of Commerce or labor organization.

In each visit, persons affiliated with the key stakeholder groups were interviewed. Personal interviews were conducted individually or in small groups, depending upon the persons being interviewed, the confidentiality of information sought, and the accessibility of the interviewees. Data collected via the interviews were recorded in written notes. In nearly all cases, cassette tape recordings were used as an additional source of documentation. Survey instruments, along with pre-addressed and stamped envelopes, were distributed to students, faculty, and mentors to supplement the on-site data collection. Based on a compilation of all of this data, each team was responsible for drafting a case study report to document what was learned in relation to the research objectives. The format for the case-study reports paralleled the research protocol used in the data collection activities. The following are the primary sections of the case studies:

• Institutional Characteristics and Mission
• Program Characteristics and Goals
• Stakeholders
• Program Components and Outcomes according to School-Based Learning, Work-Based Learning, and Connecting Activities
• Strengths and Weaknesses/Lessons Learned

Data Analysis and Reporting

Debriefing Session

After the data was collected in late October 1994, all ten data collectors gathered for a one-day meeting held during the fall conference of NCOE in Chicago, Illinois. This session was designed to assist in finishing the individual case study reports as well as to provide a forum for discussion of the major findings, conclusions, and recommendations of the study. The group discussion was facilitated by the project co-directors using a protocol developed prior to the session. The protocol involved having each research team provide a brief description of major findings, lasting no more than 30 minutes per presentation.

Following the overview given for each program, the discussion focused on identifying factors thought to be associated with exemplary work-based learning policy and practice. These factors were defined with examples shared by individual researchers, and then the group was polled to determine the incidence of the factors within particular programs. When the factors seemed to be representative of all or nearly all of the sites, the factors were defined in a way that was broadly representative of what was observed across sites. Ideas that seemed more idiosyncratic were not discarded, but recorded to assist in further interpretation of the data or in generating future research and policy questions. At the completion of the day-long meeting, the research team had generated an extensive amount of information about two-year college work-based learning policies and practices useful in addressing the study's research objectives. The final part of the discussion centered on issues and concerns evident in the work-based learning programs studied. All ideas generated by the research team were noted on flip charts to assist in data analysis during the meeting and during later report writing. The entire session was video and audio taped to assist in documenting the event. The authors of this report took primary responsibility for synthesizing and reporting the ideas generated during the debriefing session.

Preparation of this Report

Following the debriefing session, each team of data collectors finalized its case study report and transmitted the report and all information pertinent to the research to the NCRVE-University of Illinois site. Then, all case studies were organized into a common format and edited. Draft versions of each case were mailed to the field-study sites for verification of the accuracy of information. In addition, notes and audiotapes obtained during the debriefing session in Chicago were transcribed, reviewed, and used as a basis for the findings and conclusions presented in this report. A draft of the final report was shared with each member of the research team for review and critique.