Structured concept mapping is based on a three-phase model for conceptualizing program theory developed by Trochim and Linton (1986). This model suggests there are three general phases in conceptualizing program theory:
In a letter directed to the coordinators, the purpose of the study was explained, along with the basic requirements and procedures for concept mapping. A nomination form was included with the letter of invitation so that each coordinator could nominate an educator, student, and employer from his or her site who were actively involved and strongly committed to Tech Prep. (See Table 2 for a summary of stakeholder participant affiliations by site.)
Urban Schools Network Site
|
Educator
|
Student
|
Employer
|
| Nashville,
TN
|
X
|
X
|
X
|
| Raleigh,
NC
|
X
|
X
|
X
|
| Oklahoma
City, OK
|
X
|
X
| |
| Denver,
CO
|
X
|
X
|
|
| Oklahoma
City, OK
|
X
|
X
|
|
| St.
Paul, MN
|
X
|
||
| Milwaukee,
WI
|
X
|
X
|
X
|
| Omaha,
NE
|
X
|
X
|
X
|
| Washington,
DC
|
X
|
X
| |
| Detroit,
MI
|
X
|
||
| Seattle,
WA
|
X
|
X
|
X
|
| Washington,
DC
|
X
|
X
|
X
|
| Houston,
TX
|
X
|
X
|
X
|
| Las
Cruces, NM
|
X
|
X
|
X
|
| Brooklyn,
NY
|
X
|
X
|
|
| Charlotte,
NC
|
X
|
X
| |
| Philadelphia,
PA
|
X
|
X
| |
| New
Orleans, LA
|
X
|
X
|
|
| Akron,
OH
|
X
|
X
|
X
|
| Indianapolis,
IN
|
X
|
X
| |
| Mentor
Sites Portland, OR |
X |
X |
X |
| Redwood
City, CA
|
X
|
X
|
|
| Austin,
TX
|
X
|
X
| |
| Hamlet,
NC
|
X
|
X
|
X
|
| Palatine,
IL
|
X
|
X
| |
| Leonardstown,
MD
|
X
|
X
| |
| Total
|
24
|
18
|
19
|
After the letters had gone out and follow-up telephone calls had been made, a list of 86 stakeholder participants was assembled from the nominations received. Each person nominated for the study was contacted by mail, and all of the materials that needed to be completed were forwarded. Of all persons nominated, 61 returned the data collection instruments, providing a 72% response rate. These nominated persons represented 20 of the 30 NCRVE Urban Schools Network sites and all six mentor sites invited to participate in the study.
Slightly more educators participated in the study than students or employers, although the sample size for all three of the groups was sufficient. Concept mapping is conducted with a small group of experts who bring disparate, but informed perspectives. Trochim (1989a) reports that the group size for concept mapping typically ranges "from ten to twenty people" (p. 17), although concept maps can be conducted with groups of seventy-five or more. The rationale for keeping the group small, however, is that the primary goal of concept mapping is to obtain a deep understanding about how a particular group conceptualizes whatever complex phenomenon is being investigated. To do this, it is crucial to ensure the cooperation of all the participants in all phases of the concept mapping process, a process that can take each participant as much or more than two hours to complete the rating and sorting exercise. On the other hand, it is important to maintain a large enough sample to ensure disparate perspectives are represented when the concept maps are developed and interpreted. Given these parameters, the sample size for this study is optimal for determining the results for the subgroups as well as the aggregate group of all respondents.
Table 3 provides a profile of the three stakeholder participant groups. Briefly, representation by males and females is nearly equal in the study. There is not as equal a representation of other demographic characteristics (although the findings may be representative of the local consortia sites). The majority of participants are White/Caucasian and affiliated primarily with secondary vocational education. However, looking at the data closely gives a unique profile for each group.
Stakeholder |
Gender |
Race/Ethnicity |
Primary Affiliation |
Major
Responsibilities
|
| Educators (n=24) |
Male
38% Female 62% |
African
Amer. 28% Hispanic 0% Native Amer. 0% White/Cauc. 64% Asian Amer. 4% UK 4% |
Secondary
58% Postsec. 29% UK 13% |
Vocational
54%
Academic 8% |
| Students (n=18) |
Male
39% Female 61% |
African
Amer. 39% Hispanic 11% Native Amer. 0% White/Cauc. 39% Asian Amer. 0% UK 11% |
Secondary
39% Postsec. 45% UK 16% |
--
|
| Employers (n=19) |
Male
69% Female 26% UK 5% |
African
Amer. 5% Hispanic 5% Native Amer. 0% White/Cauc. 85% Asian Amer. 0% UK 5% |
--
|
Educ.
Coor./ Director 42% Owner/Bus. Mgr. 32% Public/CBO Director 16% Human Res. Director 10% |
|
|
Male
47% Female 52% UK 1% |
African
Amer. 25% Hispanic 5% Native Amer. 0% White/Cauc. 63% Asian Amer. 1% UK 6% |
Secondary
51% Postsec. 35% UK 14% (Data for educators and students only) |
Vocational
56% Academic 8% Admin. 32% UK 4% (Data for educators only) |
Note: UK indicates unknown.
First, the demographic information collected from educators shows that the majority are female, White/Caucasian, and affiliated with secondary vocational education; slightly less than one-third are African American, affiliated with postsecondary education, and engaged in administration. Students, the second stakeholder group, are similar to educators in that most are female; however, the race/ethnicity and educational affiliation of students differ from educators. About one-half of the students are minority, either African American (39%) or Hispanic (11%), and nearly one-half are attending postsecondary education. The third group, employers, is dominated by males and Whites/Caucasians. About one-half of the employer group is comprised of persons working as corporate educational coordinators or human resource directors. Another one-third of the group is composed of independent business owners or business managers, and a smaller proportion is made up of persons working for public or community-based organizations (CBOs).
An attempt was also made to draw ideas from the education reform literature such as the Project 2061 report published by the American Association for the Advancement of Science in 1989; the Curriculum and Evaluation Standards for School Mathematics also published in 1989; and the Standards Project for English/Literature Arts sponsored by the Center for the Study of Reading at the University of Illinois, the International Reading Association, and the National Council of Teachers of English. Other sources reporting academic outcomes were reviewed as well, including Kulm and Malcom (1991); Steffy (1993); and White (1994). In addition, planning documents and reports submitted by the NCRVE Urban Schools Network sites were reviewed for specific outcomes statements (e.g., NCRVE, 1993).
After conducting the concept mapping procedure, the experts also provided suggestions for improving the clarity and content of the statements. These suggestions were made verbally and/or in writing. The experts were also asked to help reduce the number of statements to a smaller number because of restrictions of the computer program for no more than 98 statements total. However, although the experts offered valuable comments regarding the content of statements, most did not eliminate statements. Consequently, the number of statements was reduced from 118 to 98 by randomly eliminating 20 statements. A list of the final 98 items contained in the "Tech Prep Student Outcomes Rating Form" is provided in Table 4. (See Appendix B for a list of the 98 outcomes statements categorized according to their predominant location in the literature.)
Student Outcome Statements
|
| 1.
create meaning from messages communicated through listening
|
| 2.
understand nonverbal communication
|
| 3.
make academic progress on grade level
|
| 4.
communicate ideas and emotions through the fine arts (e.g., art, music, dance)
|
| 5.
demonstrate consistent, respectful, and caring behavior
|
| 6.
apply basic algebra and geometry to solve technical and work-related problems
|
| 7.
organize information through the development and use of classification rules
and systems
|
| 8.
recognize the need for lifelong learning to enhance skills and learn new skills
|
| 9.
adapt to emerging technology and adjust to changing work environments
|
| 10.
exercise leadership in a variety of situations
|
| 11.
apply knowledge, skills, and learning strategies to career and life choices
|
| 12.
get along with a variety of people
|
| 13.
use appropriate and relevant scientific methods to solve specific problems in
real-life situations
|
| 14.
participate as a member of a team
|
| 15.
show good working relationships with superiors and coworkers in an occupational
role
|
| 16.
evaluate others' performance and provide feedback
|
| 17.
serve clients/customers
|
| 18.
teach others new skills
|
| 19.
resolve conflict based on divergent interests and perspectives
|
| 20.
know how to give and take instructions
|
| 21.
appreciate the diversity of values and cultural differences among people
|
| 22.
complete secondary school
|
| 23.
expand own knowledge by making connections with new and unfamiliar knowledge,
skills, and experiences
|
| 24.
communicate ideas by quantifying with whole, rational, real, and/or complex
numbers
|
| 25.
plan and work together in meetings
|
| 26.
demonstrate oral and verbal proficiency in technical communication (reports,
policies, procedures)
|
| 27.
appreciate own and others' artistic products and performances
|
| 28.
apply group problem-solving strategies
|
| 29.
use computers and other electronic technology to gather, organize, manipulate,
and present information
|
| 30.
communicate ideas and information through writing
|
| 31.
demonstrate self-control and self-discipline
|
| 32.
earn college credit in high school
|
| 33.
know employer expectations for job performance
|
| 34.
know how social, organizational, and technological systems work
|
| 35.
demonstrate the ability to be adaptable and flexible
|
| 36.
make a successful transition from education to employment
|
| 37.
make ethical decisions
|
| 38.
know own abilities, strengths, and weaknesses
|
| 39.
maintain good physical, mental, and emotional health
|
| 40.
apply logical reasoning to develop solutions to complex problems
|
| 41.
select, use, and maintain appropriate tools, information, materials, and
equipment
|
| 42.
build own self-esteem
|
| 43.
demonstrate motivation to learn
|
| 44.
use initiative, imagination, and creativity
|
| 45.
demonstrate a positive attitude toward school
|
| 46.
attend school regularly
|
| 47.
communicate ideas and information through speaking
|
| 48.
demonstrate an ability to calculate through ratios, proportions, and percentages
|
| 49.
complete postsecondary school
|
| 50.
construct meaning through reading for information, literary experience,
and to perform a task
|
| 51.
use critical thinking skills in a variety of situations
|
| 52.
use models and scales to explain or predict the organization, function, and
behavior of objects, materials, and living things
|
| 53.
use division, multiplication, addition, and subtraction with real numbers,
decimals, fractions, integers, roots, and powers
|
| 54.
achieve and maintain employability in a high-wage job
|
| 55.
articulate personal values and beliefs as they relate to a particular occupation
|
| 56.
use scientific methods to acquire information, plan investigations, use
scientific tools, and communicate results
|
| 57.
be critically aware of social issues involved in a field of interest
|
| 58.
know the history of a particular occupation
|
| 59.
observe, analyze, and interpret human behaviors to acquire a better
understanding of self, families, and other human relationships
|
| 60.
make a smooth transition from secondary to postsecondary education
|
| 61.
recognize and apply the democratic principles of justice, equality,
responsibility, choice, and freedom
|
| 62.
use the metric system and convert between metrics and traditional systems
|
| 63.
recognize the geographic interaction between people and their surroundings and
make responsible decisions for the environment
|
| 64.
design, maintain, and improve systems
|
| 65.
monitor and correct own performance
|
| 66.
use goal-relevant activities, rank them, and allocate time for them
|
| 67.
recognize varying forms of government and address issues of importance to
citizens in a democracy
|
| 68.
be dependable and punctual
|
| 69.
enter postsecondary programs without remediation
|
| 70.
prepare and use budgets, make forecasts, keep records, and make adjustments to
meet objectives
|
| 71.
use decision-making processes to make informed choices among options
|
| 72.
use research tools to locate sources of information and ideas relevant to a
specific need or problem
|
| 73.
show appropriate personal appearance and attitude
|
| 74.
acquire, store, allocate, and use materials and space efficiently
|
| 75.
have awareness of and interest in technical careers
|
| 76.
be honest and demonstrate integrity
|
| 77.
achieve certification of mastery in an occupation
|
| 78.
apply the English language correctly (spelling, grammar, structure)
|
| 79.
apply appropriate safety and environmental measures
|
| 80.
develop and follow through on individual career plans and goals
|
| 81.
be loyal to an employer
|
| 82.
gain experience in all aspects of an industry
|
| 83.
prepare and follow schedules, and manage time efficiently
|
| 84.
work under tension or pressure
|
| 85.
work without close supervision
|
| 86.
demonstrate awareness of workforce and societal trends
|
| 87.
understand the relationships between theory and practice in a technical area
|
| 88.
participate in work-based learning experiences
|
| 89.
succeed in the transition from secondary or postsecondary education to a 4-year
college
|
| 90.
recognize and apply quality standards
|
| 91.
understand the norms and values of the work culture
|
| 92.
understand how technology affects quality of life
|
| 93.
apply advanced algebra, analytic geometry, and/or calculus to solve technical
and work-related problems
|
| 94.
read and create charts, tables, and graphs
|
| 95.
prepare for direct participation in the democratic process
|
| 96.
understand the principles of competition, cooperation, and leadership in a work
environment
|
| 97.
understand and communicate in a second language
|
| 98.
recognize differences and commonalities in the human experience through
productions, performances, or interpretations
|
To assist with interpretation of the data, preliminary results of the study were presented to a small group of NCRVE Urban Schools Network participants at the March, 1995 meeting of that group in Washington, DC. At that meeting, various concept maps were presented to the participants who were asked to assist in labeling the maps and suggesting alternative interpretations of their meanings. This interpretation session was helpful in determining how the stakeholder participants were likely to interpret the final results. It was also helpful in determining how to represent the concept maps visually and verbally in this concept paper.
As completed concept mapping materials were returned, data from the rating and sort forms were entered into the concept mapping computer program called "The Concept System" (Trochim, 1989a). This program was used to aggregate the sort and rate data provided by the stakeholder participants. It "uses a combination of multidimensional scaling (MDS) and cluster analysis techniques to represent conceptual domains underlying the data" (Caracelli & Riggin, 1994, p. 142). Information from the Background Form was compiled into a spreadsheet program. One map was generated to represent the collective view of all stakeholder participants. Additional maps were generated to represent the perspectives of the three stakeholder groups of educators, students, and employers. Further information about the computation and interpretation of these maps is presented in the next section.