 |
|
Kimberly Ramsey
Cathleen Stasz
Tor Ormseth
Rick Eden
Jennifer Co
RAND
National Center for Research in Vocational Education
Graduate School of Education
University of California at Berkeley
2030 Addison Street, Suite 500
Berkeley, CA 94720-1674
Supported by
The Office of Vocational and Adult Education
U.S. Department of Education
September 1997
FUNDING INFORMATION
| Project Title: | National Center for Research in Vocational Education |
|---|---|
| Grant Number: | V051A30003-97A/V051A30004-97A |
| Act under which Funds Administered: | Carl D. Perkins Vocational Education Act P.L. 98-524 |
| Source of Grant: | Office of Vocational and Adult Education U.S. Department of Education Washington, DC 20202 |
| Grantee: | The Regents of the University of California c/o National Center for Research in Vocational Education 2030 Addison Street, Suite 500 Berkeley, CA 94720 |
| Director: | David Stern |
| Percent of Total Grant Financed by Federal Money: | 100% |
| Dollar Amount of Federal Funds for Grant: | $4,500,000 |
| Disclaimer: | This publication was prepared pursuant to a grant with the Office of Vocational and Adult Education, U.S. Department of Education. Grantees undertaking such projects under government sponsorship are encouraged to express freely their judgement in professional and technical matters. Points of view or opinions do not, therefore, necessarily represent official U.S. Department of Education position or policy. |
| Discrimination: | Title VI of the Civil Rights Act of 1964 states: "No person in the United States shall, on the ground of race, color, or national origin, be excluded from participation in, be denied the benefits of, or be subjected to discrimination under any program or activity receiving federal financial assistance." Title IX of the Education Amendments of 1972 states: "No person in the United States shall, on the basis of sex, be excluded from participation in, be denied the benefits of, or be subjected to discrimination under any education program or activity receiving federal financial assistance." Therefore, the National Center for Research in Vocational Education project, like every program or activity receiving financial assistance from the U.S. Department of Education, must be operated in compliance with these laws. |
This document is a guide for teachers and trainers participating in the initial experimental offering of a six-week "mini-sabbatical," "Designing Classrooms that Work." The mini-sabbatical is designed to help teachers learn how to make the kinds of curricular and pedagogical changes implied by reforms to integrate vocational and academic education and to improve the school-to-career transition.
The guide constitutes the basic text for the mini-sabbatical as a course of study and is supplemented by readings on curricular and pedagogical innovation. The guide explains the background and purpose of the mini-sabbatical and provides a detailed outline of activities and requirements. A companion document, Designing Classrooms that Work: Conception and Pilot Study, describes the conceptual framework underlying the design of the mini-sabbatical and reports on the initial implementation and outcomes of the mini-sabbatical, which was conducted in the summer of 1996.
The mini-sabbatical is designed for high school teachers in a variety of school-to-career programs such as career academies, cooperative education, school-based enterprises, and career focus schools. This guide is intended as a source of ideas and activities for conducting staff development for teachers, whether or not the entire six-week mini-sabbatical is implemented.
Development and testing of the mini-sabbatical at RAND was funded through a grant to the National Center for Research on Vocational Education, University of California at Berkeley, from the U.S. Department of Education, Office of Adult and Vocational Education.
The mini-sabbatical provides teachers with a unique opportunity to learn about today's workplace and draw meaningful connections between the workplace and the classroom. The goal of the mini-sabbatical is to enable teachers to acquire skills and behaviors that will (1) increase their knowledge of work practice; (2) help them create a high-quality, integrated curriculum that incorporates both domain-specific (e.g., academic, technical) and generic (e.g., problem-solving, communications) skills; (3) help them design classrooms that promote authentic learning; and (4) help them develop assessments that provide meaningful feedback to the students and teacher.
In order to accomplish this, teachers participating in the mini-sabbatical will
Almost universally, America's teachers have been trained to teach curricula that are school-based and subject-specific. But federal legislation and school reformers are urging that teachers develop and teach curricula that focus on "generic" skills such as problem solving and teamwork; integrate vocational and academic education; and emphasize "real world" applications, especially applications found in the workplace. Unfortunately, most teachers are being asked to change their practice without the requisite knowledge or the means for acquiring it (Bodilly, Ramsey, Stasz, & Eden, 1992). To make use of the workplace as a context for learning, teachers need (1) knowledge of work and work practice, (2) a new model for classroom design and instruction, and (3) the opportunity to learn and apply both.
We and our colleagues have conducted research to help meet the first two of these needs. Our study of skills requirements for work provided new insight into worksites and their potential contribution for classroom design (Stasz, Ramsey, Eden, Melamid, & Kaganoff, 1996). In addition, we developed and field tested worksite assessment methods, including cognitive task analysis and observation of skills and work context. These methods assess skills required for work at a level which can inform curriculum design. Other work addressed the need for a new model of classroom design and instruction. Two of our studies suggested that practitioners from both vocational and academic disciplines can benefit from an instructional model focused on the teaching of generic skills such as problem solving, cooperation, and dispositions such as persistence and boldness (Stasz, McArthur, Lewis, & Ramsey, 1990; Stasz, Ramsey, Eden, DaVanzo, Farris, & Lewis, 1993). This model, which we refer to as the "Classrooms that Work" model, requires teachers to assess the skills, attitudes, and knowledge required of work and to design lessons and classroom environments that support learning them.
The mini-sabbatical is designed to meet the third need, providing teachers with an opportunity to acquire knowledge of the workplace and new design models and to apply that knowledge in developing and teaching innovative curricula. The mini-sabbatical will also test the utility for teachers of the worksite observation methods and of the "Classrooms that Work" model, as well as the feasibility of requiring teachers to learn to use and apply these tools and to change their practice in a relatively short time.
The six weeks of the mini-sabbatical are divided into three two-week sections; at the end of each section, participants will have acquired important skills and experience that they can use in the future to modify their curricula and teaching. The following is a schematic view of the mini-sabbatical syllabus. A more detailed version appears in Appendix A-1:
In the first two weeks, teachers will learn ethnographic techniques for observing work in worksites and practice using these techniques to extract elements of authentic practice for potential inclusion in curricula. The worksite observation included in this mini-sabbatical emphasizes highly structured and multi-day study of the workplace. Coaching on these observational methods will continue through the week of worksite visitations. Participants' insights and conclusions about the nature of work, relations in the workplace, and tasks will then form the basis of their curriculum design work in the remainder of the mini-sabbatical. These weeks will also include an introduction to the mini-sabbatical as a whole.
In the second two weeks, teachers will apply their observations to curriculum design using the "Classrooms that Work" model. The model integrates various dimensions of the educational setting such as instructional goals, teacher role, student role, and classroom design--and encourages a consistent perspective on managing each of these elements. This model was derived from classroom observation research with teacher and student participants (Stasz et al., 1993). Teachers will use the model together with their worksite observations to develop a short curriculum unit that reflects authentic practice. Teachers will be provided the resources--time, workspace, peer collaboration, and materials--to develop curriculum. The project facilitators will coach and guide teachers to support their active construction of new professional habits of practice.
In the third and final two-week section, teachers will gain experience in teaching and assessing the newly devised curriculum. The mini-sabbatical serves as a partial "proof of concept" of the "Classrooms that Work" model. It tests whether, other things being equal, strengthening a classroom's authentic work context, coupled with redesign of the curriculum in alignment with work-inspired goals, will produce a classroom that is more successful in imparting "generic" vocational skills and attitudes.
The mini-sabbatical follows a professional, adult training model common in industry that relies strongly on the attitudes, knowledge, skills, and experiences that adult participants bring with them. Throughout the mini-sabbatical, trainers will repeatedly call on participants to reflect upon the ways in which their new insights into work and the classroom impact and interact with their own personal perspectives on, and prior experiences with, teaching and learning.
This section contains suggestions for implementing a "Classrooms that Work" mini-sabbatical. We write it with the knowledge that each implementing organization will make changes to fit their resource capabilities and available expertise.
Besides actual instruction, five major functions are important to staff. They include coordination of the following: (1) school and classroom arrangements; (2) teacher recruitment and selection; (3) worksite and mentor recruitment; (4) student recruitment; and (5) administrative arrangements such as clerical support, payroll, and so on. We address the first four functions below. The fifth, administrative arrangements, depends entirely on the implementing organization.
If you are working within a school district, include the local teachers' union and staff development department early in the negotiations. Discuss the recruitment process for teachers and students, stipends, classroom requirements, campus security, informed consent, and permission and liability issues.
This professional development project requires participation of eight to ten active teachers (including one to two teacher trainers), 60 students, and eight to ten worksite mentors. We suggest entering into letter agreements with all participants. Below we detail plans for teacher recruitment and selection.
The following is a proposed breakdown of practitioner participants: eight active academy or magnet teachers and two active teacher trainers. Active teachers will be a mix of teachers with academic, vocational, and technical teaching assignments. Active teacher trainers will come from schools of teacher training, vocational instructor training programs, and school district consultants assigned to staff development. We suggest beginning recruitment of participant teachers and setting stipend amounts in January and February before they commit to other summertime activities.
There are two ideal teacher candidates: one with a positive disposition to the instructional practices that we believe are important, the other a "master" teacher with a neutral attitude about integration but an interest in exploring the possibilities. Don't bother working with persons who are overtly hostile to what the training effort proposes to do.
As part of the recruitment process, teacher candidates complete a survey of their backgrounds, typical practices and preferred ways of working within the educational system (e.g., level of comfort with crosscurricular planning), and desired worksite assignment (see Appendix B-1). Candidates will also submit a work sample: the sample should be a project or instructional unit designed by the teacher that they have found to engage students. Results of the survey and an individual interview are part of the selection process. The interview will provide the opportunity for candidates to question the project staff, discuss the work sample, and clarify survey results. An ideal candidate will want to acquire new skills, to work collaboratively with other teachers, and will agree that worksite observation is useful to this end.
The project director and each teacher will discuss plans for developing a curricular unit and select an appropriate worksite assignment accordingly. The following decisions need to be made about the curricular unit: (1) whether the participant will perfect a project or study unit previously taught or create a new project or study unit; and (2) specify which study units and course will contribute domain-specific content to the project (e.g., engineering aspects of design architecture as part of an Engineering Principles course; project management using GIS application as part of a Technology course; Pythagorean theorem application to engineering and construction as part of an Algebra I course). The project director contacts worksites to effect an appropriate placement.
Contact worksites and recruit ten worksite mentors. An ideal worksite is one that is a current site for work-based learning or internships with established links to schools and assigned mentors. To identify such a site, contact a local high school with an established work-based learning program. Through the high school's program, select candidate worksites and mentors. City or county government agencies, public works, and utilities are an excellent and often overlooked source for worksites and mentors, particularly in technology- and science-related subjects.
A suggested entree protocol includes the following: (1) recruitment letter specifying the desired observation; (2) formal meeting with mentor or other workplace contact; (3) formal agreement with the worksite and mentor. Mentor participation involves the following: serve as key informant during worksite observation week, and serve as contact for teacher participants' questions during weeks 3 and 4 (see Sample Worksite Letter in Appendix B-2). We also invited mentors to serve as reviewers for presentations of the draft curriculum unit (close of week 4) and the final curriculum unit by teachers and students (close of week 6).
Student participation is essential to rigorous field testing and evaluation of integrated curriculum. School administration and counseling staff will help to identify prospective students. We sought and received written permission from parents for their child's participation in the project. We advocate paying students stipends for their participation: students are required to complete a daily survey and extensive journal to support field testing the curricular unit and evaluating student learning and satisfaction. (See Student Guide, Appendix B-3.)
We suggest a sample of students that reflects the diversity of the local high school population--for example, multiple racial and ethnic groups; English proficiency; gender balanced; and previous participation in the range of available programs such as college preparatory, general, vocational, and special education. Students are presurveyed about their education background and their experiences working on projects, in teams, and with various technologies.
The main body of this guide is organized into three sections corresponding to the major parts of the mini-sabbatical. Each section has two main elements: an introduction and a description of unit activities. In addition to the syllabus, appendices document schedules, protocols, evaluation instruments, and other materials to support implementation.
The purposes of the first week are generally to prepare teachers to perform successfully in the mini-sabbatical and, more particularly, to perform successfully in the second week, when they will be required to observe worksites and derive knowledge that can be applied in the development of new curricular units.
In the first week, participants will acquire knowledge and skills of three sorts:
Participants also initiate a process of reflection on and integration of new concepts and attitudes into the perspectives that each participant brings to the mini-sabbatical. This process will be facilitated and documented through the regular use of journals.
Prior to the first day of activities, mini-sabbatical participants will have received and read selections from the research report, Classrooms that Work (see Syllabus, Appendix A-1). The reasons for doing so are twofold: (1) to orient participants to the conceptual basis of the mini-sabbatical; and (2) to set high expectations for engagement and participation in the course.
The purpose of these two activities is to enable participants to begin learning about each other.
The purpose of this unit is to provide participants with a common understanding of the mini-sabbatical, including the origins of its development, its organization, and their roles and responsibilities.
Participants will have read selections of research reports as background and will listen to a presentation by research staff. The presentation will follow the "Introduction to the Mini-Sabbatical" beginning on page 1, and will discuss the current direction of vocational education research, the theoretical basis for the mini-sabbatical, and findings from the research projects out of which the mini-sabbatical grew. The trainer will then explain the design of the mini-sabbatical and lead a discussion of the syllabus.
Materials provided: journal notebook
This activity will explain the purpose of the mini-sabbatical journal and provide participants with a chance to practice writing a journal entry.
One purpose of journals is to assist in the integration of new understandings with one's personal perspective and to account for personal theories that participants hold.
Periodically, the trainers will collect journals. Privacy of authors will be protected, and confidentiality will be maintained in reporting and dissemination.
Participants write in their journals to the following prompts:
An open discussion of participants' questions and concerns follows.
Note: Throughout this guide, we provide multiple journal prompts. In actuality, only one or two such prompts, selected to reflect the direction of the preceding discussion, will be used.
This activity introduces mini-sabbatical participants to the "Classrooms that Work" (CTW) model. Teachers will have read background materials (see Week 1 Syllabus, Activities 1.0-1.5, for these readings) and then hear a presentation. (Slides of the presentation were selected from the briefing, "Designing Classrooms that Work," which can be found in Appendix A-4.) They will discuss the presentation in class. This presentation describes the research project that led to the development of the CTW model, focusing on two classrooms taught by the same teacher, contrasting instructional goals, classroom design, teacher roles, school context, and student perceptions.
At the conclusion of the presentation and discussion, teachers will spend 15 minutes journalling in response to the following prompts:
From this unit, participants will learn (1) the constructivist perspective and its meaning for classroom practice; (2) cognitive science notions about the difference between domain-specific and "generic" skills, and the importance of "situated" practice; (3) the dimensions of classroom environments that are successful in imparting generic skills; and (4) the close relationship between the dimensions. Successful classrooms integrate appropriate goals, tasks, roles, and routines, typically around a project of some sort. The CTW model involves integrating authentic or "work-inspired" goals, tasks, roles, and routines, typically around a project or investigation.
As the presentation of the model proceeds, project faculty will make connections back to the discussion of authentic practice, high quality, and teachers' personal theories that were introduced and discussed in Activity 1.2 (Overview of the Mini-Sabbatical).
This activity is designed following a "jigsaw" model, where participants share responsibility for presenting information to each other.
Participants pair up by subject taught. Each pair is given a case study from the "Classrooms that Work" report that relates to their specialization. They are given 20 minutes to review the case and to discuss key points. They focus on finding specific examples of the dimensions in the "Classrooms that Work" model, and on identifying the "authentic" culture of practice developed by each teacher and his or her students.
In a trainer-led discussion, participants share what they have learned, following the order of the case studies in the report. The following are some prompts for leading the discussion:
Materials provided: Zulu love letter pin kits
The purposes of this activity are to focus on the use of teamwork as a generic skill and to provide a specific basis for a discussion of high-quality learning.
Participants will pair and make Zulu love letter pins.[1] Each team may accomplish the task in any way they see fit. They must keep careful notes on their goals (what they want to get out of the experience) as well as the process, including difficulties, breakthroughs, and any new awareness and/or insights about learning and designing learning. Then the teams develop a two- to three-minute presentation to the group. The mini-sabbatical trainer models appropriate teacher roles such as coaching, scaffolding, and fading.
After presentations, participants engage in a trainer-led discussion:
Participants write for about 15 minutes on the following prompt (the questions in Activity 1.6 can also be used):
The purpose of this activity is to help participants think about what constitutes good teaching from the perspective of a student engaged in learning. Participants will read an article by Collins and Frederiksen, "Five Traits of Good Teaching: Learning, Thinking, Listening, Involving, Helping"; then they will attempt to apply the readings to the analysis of their own learning process. Finally, they will adapt the learning activity for a set of hypothetical students.
Discussion will test the participants' understanding of their readings:
Using the CTW model and insights from the discussion, each participant then outlines a classroom design for a project requiring his or her own students to make Zulu love letter pins.
Participants discuss their learning in light of the readings:
Finally, participants should review their previous journal entries ("write a note to your colleagues back at school on what makes a classroom work") and reflect on making the Zulu love letter and the discussion that followed. Then they should enter any modifications to what they had written earlier, if needed.
Materials provided: blank field notebook, sample fieldnotes, worksite videos, and videotape player and screen
The purpose of this activity is to introduce participants to the skills they will need to perform, analyze, and document worksite observations in the second week. Participants will hear presentations on the goals and techniques of worksite observations and have the opportunity to practice prior to placement in the field.[2]
Presentations by the mini-sabbatical trainers will address (1) authentic practice, work context, and the rationale for worksite observations; (2) understanding work from workers' perspectives; (3) techniques for observing and documenting work; (4) types of tasks suitable for the design of high-quality learning experiences; and (5) logistics of workplace observation for the second week (e.g., assigned mentor, schedule, directions and parking). Briefing slides for these presentations can be found in Appendix A-2.
Materials provided: "The Job Study," a tutorial and reference guide for teachers (see Appendix A-3); and a blank field notebook. The project faculty will arrange for each worksite mentor to issue teachers any special clothing or gear required at the worksite and to explain its use.
The goal for Week 2 is for teachers to observe problem solving in worksites. Teachers are matched to worksites and mentors based on their proposed curriculum units. The project faculty will encourage teachers to be inquisitive and to take the worker's lead in directing their observation. The faculty will coach teachers, as needed, on observation techniques.
Five days are required to accomplish this stage of the mini-sabbatical. On the first day, teachers travel to selected worksites and meet their mentors, department supervisor, and other staff members for a general orientation to the workplace.
On day two, teachers observe work tasks, beginning with routine and frequently performed tasks and then shifting to infrequent and complex ones. Routine tasks form a background of activity against which less frequent, but more complex and system-oriented aspects of the job can be understood.
The third day is devoted to summarizing fieldnotes and debriefing with other teachers and the project faculty.
Teachers return to the worksite on day four to observe start-up routines and follow up on task observations begun on the second day. The remainder of the fourth day is reserved for brief 20-minute interviews with a department head and trainer responsible for staffing the department. From these interviews, teachers appreciate the connections between the department and the larger organization, performance expectations for employees, and staff training.
On the fifth day, teachers return to the classroom for a final round of debriefing and begin writing up a summary of the department, the tasks performed, authentic problems encountered, and how workers go about solving problems. Authentic problems and the work context form the basis by which teachers develop classroom learning experiences.
The goal for Weeks 3 and 4 is to learn how to design and assess "classrooms that work" based on results of the work observations and analysis in Week 2. The third week begins with discussions on approaches and options for curriculum and assessment design. At the end of the third week, each participant will be asked to present a first draft of his or her curriculum and associated assessment plan to the other teachers. The project faculty will provide coaching and direct teachers to use each other as resources for curriculum development. During the fourth week, teachers refine their projects or investigations, learn about students assigned to their experimental class, prepare their classrooms, and make a presentation of their experimental classroom design to other teachers, project faculty, and worksite mentors.
During this phase, some participants may discover that they wish to observe particular aspects of the worksite again. It may be possible to arrange for additional visits or to contact the appropriate worksite mentors or department supervisor for follow-up discussions.
The first step in the process of moving from worksite observation to instructional design is to produce a useful written summary of authentic practice based on the results of the job study. To do this, teachers make their first critical transition: from job tasks to authentic problems.
Faculty will brief teachers on authentic problems and authentic cultures of practice using as an example a written description of services for disabled travelers, "A Day in the Life of a Tour Manager for Handicapped People." The subject matter of the example has implications for the integration of academic science classes and the transportation and health care industries, providing a basis for making the distinction between "authentic" science and "school" science. Teachers will discuss examples from their own job study where distinctions between disciplinary and authentic practice become obvious.
Following this, a trainer-led discussion on the "disabled travelers" example will address the following dimensions:
Then, teachers will write a summary of authentic practice based on their worksite observations. The authentic practice summary will address the above dimensions. First, they will work individually for one hour, and then in same-subject pairs (e.g., math, English, science, and technology) for another hour. The written authentic practice summaries will then be exchanged for comment by still another teacher.
In this activity, teachers explore alternative approaches to curriculum development and integration. Participants will read and discuss selected modules from Getting to Work: A Guide to Better Schools to gain an appreciation for the breadth of alternatives[3] and the advantages and disadvantages of each approach.
Module 2, Units 2 and 3 are employed as references that teachers may use to learn more about different industry frameworks and examples for designing curricula based on authentic problems (e.g., All Aspects of the Industry, Unit 3-8; Hoachlander's Alternative Aspects of Industries, Unit 3-9; and Industry Themes Used for Integrated Projects, Unit 3-10).
Finally, each participant must choose one approach for the curricular unit that he or she is developing; the approach may be a novel or synthesized one. Participants will describe the selected approach and the rationale in their draft instructional design.
This activity is the second critical transition point in the mini-sabbatical. Teachers shift their understandings about authentic problems and authentic culture of practice to instructional design that reflects the CTW model.
The activity begins with a discussion that again uses the disabled travelers example. After review of the authentic practice summary, elements of the instructional design for a hypothetical classroom are discussed: instructional goals, classroom design, teaching methods, and alignment with official curriculum. The example on the next page shows a completed document for the disabled travelers project that can be used in the discussion.
After the briefing, teachers begin work on their own draft instructional designs. Teachers must build their new curricula around a project or investigation that follows authentic practice and solving authentic problems. Teachers will use the CTW instructional model--with a modification of the "school context" dimension to that of alignment with official curriculum--as a framework.
Participants will produce a written (and perhaps drawn) document that explains their instructional design, using the template in the following example (included in Appendix B-1).
INSTRUCTIONAL DESIGN Task: Determine the suitability (e.g., physical, social, and technical) of the Metro Red Line for use by a person dependent on an oxygen support system. Instructional Goals: Generic: Generate and evaluate assets Disposition: Persistence, question authority Domain skill: Human biology of oxygen use, transportation of elemental oxygen, research organization, research theory Classroom Design: Culture of practice: Students are consultants hired to lobby LACMTA to accommodate dependent persons Product: Develop intervention or briefing speech to the MTA board Teacher role: Supervisor to the consultant Team: Scientist, writer Teaching Methods: Coach, model research, reinforce continuum of expertise, exploratory learning Alignment with Official Curriculum: Biology Composition
This activity introduces teachers to assessment options. Teachers will read Getting to Work: Module 4, "A Practical Guide to Alternative Assessment" and "The Range of Assessment Strategies" (Appendix A-5).
The activity will open with a discussion of principles and types of assessments.
Teachers will review examples and discuss them. Following this background discussion, the strengths and weaknesses and evaluation criteria for assessments will be covered. The activity closes by turning to applications for curriculum units.
In a follow-up session, teachers will share and critique the assessments they design for the experimental curriculum unit.
This is a critical point for the mini-sabbatical. Teachers begin to explore their instructional and curriculum design skills and habits of practice. Teachers will work at their own pace and with products clearly understood: draft presentation of the instructional design due at the close of Week 3 and draft presentation of the curriculum and assessment plan due at the close of Week 4. Faculty monitor teachers' progress and provide support as needed.
In this activity, which will be repeated as Activity 4.3, participants receive feedback from peers and faculty that will help them improve their classroom design plans.
Teachers will be divided into two groups. Each member of a group will spend five minutes describing his or her draft classroom design. A group facilitator will elicit comments for another five minutes. When everyone has had a turn presenting, the entire class will reconvene for a trainer-led debriefing.
Teachers will need to have answers to the following questions:
The purpose of this activity is to provide participants with an understanding of what action research is and how it will be used during the fifth and sixth weeks when they experiment with teaching and assessing their CTW curricular units. Participants will read Bullough and Gitlin, Chapter 8, "Action Research." The faculty will lead a discussion on the value added to the mini-sabbatical by action research, and draw out teachers' expectations relative to sharing results. The faculty will urge teachers to tell their own stories in their professional journals and at professional meetings.
As part of action research, participants will make daily journal entries for the remainder of the mini-sabbatical. These will help capture teacher participants' new understandings and concerns.
Before moving to the campus classrooms, faculty and teachers will discuss location and parking directions, room assignments, getting students to the right places, schedule, and so on.
There will be a general discussion on the feasibility of setting up workplaces in classrooms and classroom and curriculum design aspects that might be altered without compromising essential aspects that facilitate learning.
Teachers will review student pre-course survey results for students assigned to their class.
This presentation is more formal than the one in Activity 3.6. Each participant will present his or her curricular unit and assessment plan before a panel of peers, mentors, and the mini-sabbatical project team. A school administrator may be included on the panel as well.
This time, emphasis will be on design of activities and resources for actually implementing the curriculum. Teachers will need to address the following questions:
The teacher will complete the "Curriculum Revision Log-Baseline" (see Appendix B-1) as a record of their curriculum design at the beginning of the teaching phase of the mini-sabbatical.
The final two weeks of the mini-sabbatical involve teachers pilot testing the curriculum unit. The goal is for teachers and students to take center-stage in an exploration of the issues involved in implementing "Classrooms that Work."
Each teacher is assigned seven to eight students who are in class for two hours per day. Teachers and students will complete daily journals and short surveys about the learning experience. Some classroom activity will be videotaped. The videotapes will be shown to teachers daily to give them feedback on the progress of their units and to facilitate reflection on and transformation of their habits of practice.
The faculty will circulate throughout the classrooms to observe students and teachers. Faculty will focus their observations on aspects of the CTW instructional model. On the final day of the mini-sabbatical, teachers will make a final presentation of their instructional design. The outcome for each teacher is presentation of a fully specified instructional design and curriculum unit that reflects authentic practice.
In Weeks 3 and 4, teachers have already designed initial versions of their curricular units and instructional design. The pilot will enable teachers to refine their designs. Daily assessments will be used to capture introduced alternative perspectives and carefully document the classroom activity. These include (1) what students learned (student journal); (2) what activities students involved themselves in during classroom sessions (student activity report); (3) classroom dynamics (teacher journal and coaching panel); and (4) planned classroom and project/investigation activities (revision log). The presentation of the final design is the formal close of the mini-sabbatical.
The curriculum documents, journals, and survey data also serve the purpose of inquiry into school-to-career and integration reforms. The mini-sabbatical project team will use this data to evaluate what the mini-sabbatical accomplishes in the following areas: (1) providing an alternative professional development model for school-to-career and integration reforms; (2) developing technical assistance methods for introducing constructivist teaching and learning into high schools; and (3) better understanding tradeoffs between the "official" curriculum (as represented in standards or state frameworks), the "implemented" curriculum (what teachers develop in the mini-sabbatical), and the "attained" curriculum (what students learn).
Materials provided: Student handbooks, notepaper, pens, chart paper for examples, blank student checklist forms
On the first day of the class pilot test, we begin with an opening assembly. It is an opportunity for all participants in the pilot test to become clear about the purpose of the mini-sabbatical and their part in it. The assembly agenda includes mutual introductions of mini-sabbatical project staff, teachers, and students. Students will receive their class assignments, meet their class teacher, and learn how to complete journal entries and checklist activity reports.
On subsequent days, students will report directly to their teachers. Mini-sabbatical faculty and teachers will continue to hold brief meetings at the beginning of each day. Announcements, modifications, and organizational questions will be handled at this time.
Training for completing the journal and checklist report is as follows. Discuss the purpose of the journal and checklist. Ask the following questions:
Students should understand that their daily reports and journals are important to the project. The mini-sabbatical is intended to help teachers develop some new classroom methods, and student feedback is needed to help determine how new methods work for students.
Explain identifiers used on journals and checklists: teacher name and ID, and student name and ID. Students then practice journalling. The subject is "I am a student" and each student is asked to respond to the following prompts:
Students then engage in open discussion of the journal assignment. Working in groups, students discuss the assignment, answer each other's questions, and list questions that the group is unable to answer.
Someone from each group will ask the group's question(s) to the full class; the mini-sabbatical leaders will join the discussion at this point.
After journalling, students will practice writing checklist reports. First they will discuss the purpose. Leaders will distribute sample reports and blank practice forms and ask students to review the sample "activities report" by sections:
Using a blank report, the project faculty will offer examples and ask students to "report" the activity--for example, draw examples from the CTW report.
At the end of each class session, students will gather in a common area for reflection and discussion. Each student will make a one-page journal entry and complete a checklist report.
Teachers are also "learners" during the pilot test. They will address this experience in their own journals. Teachers will make daily entries of two to three pages (10-15 minutes of reflection and writing) at any time during the day.
The journals have several purposes. One is to encourage teachers to reflect on the experience of trying to implement the curriculum they have designed. Another is to provide teachers a regular opportunity to record their impressions and reactions to new types of teacher roles. In addition, teachers' journals will be useful to the project staff as we provide feedback and when we attempt to evaluate the success of the mini-sabbatical and consider how to improve it.
The journals can take any form and may contain any content that teachers wish. In other contexts, such journals have been used to move across several lines of thought:
Typically, one cannot move through all these stages of thought on every topic. Sometimes one can describe an activity but not evaluate it; or one can describe and evaluate it, but not diagnose reasons for success or failure, or figure out what to do differently in the future.
In terms of topics, it may be useful to think of the CTW model. For instance, the teachers may focus on whether students are using and learning generic skills and attempt to record evidence of such learning.
After journalling, teachers will practice completing the curriculum revision log (see Appendix B-1). The goal of the logs is to track changes in the specification of instructional goals, content, resources, learner activities, and organizational support. Teachers will complete a detailed baseline survey record on the first day, and submit daily revision logs thereafter.
In the training session on keeping revision logs, teachers will discuss the purpose and format of the logs. Leaders will distribute a sample log and blank practice forms and ask teacher participants to review the sample "revision log" by sections:
Using a blank report, the project facilitator will offer examples and ask teachers to "report" the activity (e.g., draw examples from the CTW report).
Each section has a rationale column. Participants will practice writing a rationale note for two to three examples and discuss what constitutes a "good" rationale note.
Beginning the second class day and continuing daily, the project staff will select interesting video segments from the day's class. Teachers will spend 15 minutes viewing the selected video segments followed by 15 minutes of (voluntary) discussion of observations.
This activity is based on the concept of filmmakers' use of dailies to know what they "have" in order to make revisions. Alternatively, some teachers may want to use video segments in the way that coaches and athletes use game films to critique and improve their play.
Observations should focus on the following:
This activity is repeated daily for the remainder of the mini-sabbatical. Based on the day's events in the classroom, teachers make adjustments in their instructional design and curriculum unit. The final 75 minutes of the day is set aside for this purpose. The project faculty will circulate throughout classrooms to assist teachers.
On the very last day of the mini-sabbatical, students, teachers, and project staff convene for a final presentation. Teachers present final versions of their instructional design and curricular unit. In this iteration, they discuss the technical and professional judgments involved in developing curricula.
The audience for the performance may also include worksite mentors; school, community, and district guests; guests of mini-sabbatical project faculty; and guests of teachers.
Bodilly, S., Ramsey, K., Stasz, C., & Eden, R. (1992). Integrating academic and vocational education: Lessons from eight early innovators (NCRVE Document No. MDS-287; RAND Document No. R-4265-NCRVE/UCB). Berkeley and Santa Monica: National Center for Research in Vocational Education, University of California at Berkeley, and RAND.
Stasz, C., McArthur, D., Lewis, M., & Ramsey, K. (1990). Teaching and learning generic skills for the workplace (NCRVE Document No. MDS-066; RAND Document No. R-4004-NCRVE/UCB). Berkeley and Santa Monica: National Center for Research in Vocational Education, University of California at Berkeley, and RAND.
Stasz, C., Ramsey, K., Eden, R., Melamid, E., & Kaganoff, T. (1996). Workplace skills in practice: Case studies of technical work (NCRVE Document No. MDS-773; RAND Document No. MR-722-NCRVE/UCB). Berkeley and Santa Monica: National Center for Research in Vocational Education, University of California at Berkeley, and RAND.
Stasz, C., Ramsey, K., Eden, R., DaVanzo, J., Farris, H., & Lewis, M. (1993). Classrooms that work: Teaching generic skills in academic and vocational settings (NCRVE Document No. MDS-263; RAND Document No. MR-169-NCRVE/UCB). Berkeley and Santa Monica: National Center for Research in Vocational Education, University of California at Berkeley, and RAND.
[1] These inexpensive beading kits are commonly available in craft stores. Any craft kit for an unfamiliar folk art will produce the desired effect.
[2] We suggest that teachers observe work at the training site. The idea is to facilitate practice in observing unfamiliar activities. Examples of assignments that were available at RAND: electrician, computer troubleshooting and repair, and public relations.
[3] We limit discussion of alternative approaches to those which can be accommodated by changes initiated by one or two teachers, rather than approaches that require organizational redesign and adoption by entire departments or schools.
[4] The mini-sabbatical project leaders collect, review, and summarize the daily journal entries. The privacy of student authors and confidentiality will be maintained.
In order to preserve graphics and page layout, three of the appendices are available only in Adobe Acrobat format. View Acrobat files with the Acrobat Reader, available for free downloading from Adobe at http://www.adobe.com/prodindex/acrobat/readstep.html.
This syllabus summarizes the scheduled activities of the mini-sabbatical. It is organized by day (i.e., by class meeting) and assumes a four-day week and six to eight hours of training time each day. The syllabus indicates the day's chief topic or activity, what you should read to prepare for class, and what you will do during class. The list of required readings is found after the description of daily activities. If an assignment is due (for instance, if journals will be collected) on a given day, that is indicated in bold italic. For bibliographic information, see the "Reading List" that follows this syllabus.
| Read: | Collins, Brown, and Holum, "Cognitive Apprenticeship: Making Thinking Visible" Classrooms that Work, Summary and Chapters 3-6 |
| Journal: | Fifteen minutes on topic |
| Read: | Collins and Frederiksen |
| Journal: | In response to provided prompts |
| Journal: | In response to discussion questions about in-class presentation |
| Due: | Practice fieldnotes |
| Due: | Journal entries from Week 1 |
| Journal |
| Journal |
| Make fieldnotes |
| Journal |
| Summarize fieldnotes |
| Journal |
| Make fieldnotes, conduct interviews |
| Journal | |
| Write: | Begin summary of authentic practice |
| Due: | Journal entries from Week 2 |
| Read: | Getting to Work: Module 2, "A Day in the Life of a Tour Manager for Handicapped People," Sections 1, 3, 4, 6, 8, 16 |
| Write: | Finish summary of authentic practice |
| Journal | |
| Due: | Copy of fieldnotes from Week 2 |
| Read: | Getting to Work: Module 4, "A Practical Guide to Alternative Assessment" |
| Journal | |
| Due: | Summary of authentic practice |
| Journal | |
| Due: | Explanation of curricular approach |
| Journal | |
| Due: | Draft curriculum design and assessment plan (presentation to peers and faculty) |
| Journal | |
| Read: | Bullough and Gitlin |
| Journal |
| Journal |
| Journal | |
| Due: | Curriculum design presentation |
| Revision Log |
| Journal |
| Revision Log |
| Journal |
| Revision Log |
| Journal |
| Revision Log | |
| Journal | |
| Due: | Journal entries for Week 5 |
| Revision Log |
| Journal |
| Revision Log |
| Journal |
| Revision Log |
| Journal |
| Journal | |
| Videotape | |
| Due: | Final presentation of curriculum design and assessment (to peers and faculty) |
| Due: | Journal entries for Week 6 |
| Due: | Revision logs for Week 6 |
A day in the life of a tour manager for handicapped people. (1989). Excerpt from D. W. Howell, Passport: An introduction to the travel and tourism industry. Cincinnati, OH: South-Western Publishing Company.
Bullough, R. V., & Gitlin, A. (1995). Becoming a student of teaching: Methodologies for exploring self and school context (pp. 179-202). New York: Garland Publishing.
Collins, A., Brown, J. S., & Holum, A. (1991, Winter). Cognitive apprenticeship: Making thinking visible. The American Educator, pp. 6-46. (Published by The American Federation of Teachers.)
Collins, A., & Frederiksen, J. (1989). Five traits of good teaching, unpublished manuscript.
Herman, J., Aschbacher, P. R., & Winters, L. (1992). A practical guide to alternative assessment. Alexandria, VA: Association for Supervision and Curriculum Development.
MPR Associates. (1995). Getting to work: A guide for better schools. Berkeley: National Center for Research in Vocational Education, University of California at Berkeley.
Stasz, C., Ramsey, K., Eden, R., DaVanzo, J., Farris, H., & Lewis, M. (1993). Classrooms that work: Teaching generic skills in academic and vocational settings (NCRVE Document No. MDS-263; RAND Document No. MR-169-NCRVE/UCB). Berkeley and Santa Monica: National Center for Research in Vocational Education, University of California at Berkeley, and RAND.
Case studies are Summary and Chapter 3, pp. xiii-xxiii and 21-59; Chapter 4 (English), pp. 60-82; Chapter 5 (Electronics), pp. 83-98; and Chapter 6 (Industrial Arts), pp. 99-104.
Week Location Days and Hours 1 Training Site Mon.-Wed., 9-3 2 Worksites Mon.-Tues.-Thurs., TBA Training Site Wed., Fri., 10-2 3 Training Site Mon.-Thurs., 9-3 4 Training Site Mon.-Wed., 9-3 Local High School Thurs.-Fri., 9-3 5 Local High School Mon.-Thurs., 9-3 6 Local High School Tues.-Fri., 9-3
Week, Day Unit Approx. Time Facilitator(s) Week 1, Day 1 1.1 45 minutes 1.2 20 minutes 1.3 60 minutes 1.4 90 minutes 1.5 60 minutes Week 1, Day 2 1.6 45 minutes 1.7 20 minutes 1.8 60 minutes 1.9 180 minutes Week 1, Day 3 1.9 Full day Week 2, Day 4
(worksite)2.1 Fieldwork Week 2, Day 5
(worksite)2.1 Fieldwork Week 2, Day 6
(training site)2.1 Full day Week 2, Day 7
(worksite)2.1 Fieldwork Week 2, Day 8
(training site)2.1 3.1
150 minutes 150 minutes
Week 3, Day 9 3.1 90 minutes 3.2 135 minutes 3.3 75 minutes Week 3, Day 10 3.4 240 minutes 3.5 75 minutes Week 3, Day 11 3.5 Full day Week 3, Day 12 3.5 180 minutes 3.6 150 minutes Week 4, Day 13 4.1 60 minutes 3.5 255 minutes Week 4, Day 14 4.2 75 minutes 3.5 240 minutes Week 4, Day 15
(move to classroom sites)3.5 Full day Week 4, Day 16 3.5 180 minutes 4.3 150 minutes Week 5, Day 17 5.1 90 minutes 5.2 60 minutes 60 minutes Class 5.4 60 minutes 5.5 90 minutes Week 5, Day 18 5.1 5 minutes 120 minutes Class 5.2 45 minutes 5.4 45 minutes 5.5 75 minutes Week 5, Day 19 5.1 5 minutes 120 minutes Class 5.2 45 minutes 5.4 45 minutes 5.5 75 minutes Week 5, Day 20 5.1 5 minutes 120 minutes Class 5.2 45 minutes 5.4 45 minutes 5.5 75 minutes Week 5, Day 21 5.1 5 minutes 60 minutes Class 5.2 45 minutes 5.4 45 minutes 5.5 75 minutes Week 5, Day 22 5.1 5 minutes 60 minutes Class 5.2 45 minutes 5.4 45 minutes 5.5 75 minutes Week 5, Day 23 5.1 5 minutes 120 minutes Class 5.2 45 minutes 5.4 45 minutes 5.5 75 minutes Week 5, Day 24 5.6 120 minutes All (videotape) 60 minutes Feast 120 minutes Close down Total training days: 24 Total activity units: 25 Total trainee contact hours: 144
The tutorial is an adjunct to discussions and practice sessions. It was designed to provide a ready reference while teachers are in the field conducting and documenting worksite observation.
For the outsider, another person's job is an incredible blur of people, space, time, and tasks. In our previous research, we constructed a common-sense and low-inference worklife framework to appreciate a job, to organize observations, and to accomplish the descriptive task of detailing frontline work (Stasz et al., 1996). The framework is straightforward and common to all jobs: there is a startup period followed by everyday routines, and each worker is involved in everyday relations with others. Embedded in these three aspects are skills, tasks, and work context.
We blend the worklife framework with what we know about job design to develop a plan for studying various job arrangements. We know that jobs are socially organized and physically situated in particular ways. Thus, we develop different study plans for jobs involving individual work where workers convene in a common space, jobs that involve teaming, and jobs that require workers to be highly mobile or itinerant.
On Day 1, you will meet your worksite mentor. To facilitate scheduling formal interviews, the mentor will arrange introductions with (1) the department manager and (2) the designated trainer for the department or work unit. For small departments, such positions are frequently folded into a single job.
Teacher-observers target for study a job that is related to your proposed curriculum unit. Over the course of the week, you will come to understand the target job. Specifically, you will focus on (1) the social setting for the work and (2) the frequent and critical tasks assigned to the job. Ask the mentor who has agreed to serve as your key informant to help you determine the best approach to accomplish observation of these tasks. The mentor may refer you to the department supervisor and other workers.
Planning task observation encompasses understanding the range of tasks assigned to a job. You need to observe enough workers to exhaust observation of frequent and critical tasks assigned to a job and to witness the same tasks performed by several different workers. You may observe differences in the commission of a task based on expertise of the workers or based on the type and complexity of the problem presented. You will need to "follow" the task to appreciate the complexity of the work and identify high-quality lessons that add value to student internships. Together, the strategies permit one to observe all the workers involved, their positions in the firm, and their relationship to tasks.
After you have determined the "right" job to study, ask for a tour of the immediate areas where workers congregate and where the job is typically performed.
This is a good opportunity to learn names, titles, and to figure out various roles and reporting relationships. You may also have some time to see the "tools" of the work and to acquaint yourself with typical clients. Be prepared to explain the purpose of the mini-sabbatical and the job study to all you meet, including clients. Even though your purpose is to study the work, if observation involves your contact with clients, they have the right to exclude you from the commission of their services. You need not be overly concerned, however, since it is our experience that most workers and clients are happy to be part of efforts to improve education for young people.
When target jobs convene workers in a common space and assigned to specific tasks, we suggest the following Day 2 schedule. With fairly routinized and assigned tasks, the observer is able to witness each task on the average of eight to ten times during the morning. After lunch, the observer--with knowledge of the task and its technical requirements--refocuses on everyday relations involved in accomplishing tasks.
On Day 3, teacher-observers take a break from the field, returning to the classroom to file fieldnotes, debrief and clarify, reflect, and plan.
On Day 4, the teacher-observer returns to the worksite at the beginning of the workday to observe tasks and relationships required to initiate work. We purposely delay this observation until the observer is sufficiently familiar with the work, frontline workers, and other department staff. With completion of task observations, the teacher-observer conducts formal interviews, as we discuss below.
When jobs involve teaming, we revise the above approach. Teaming complicates understanding the worksite. This is especially true when various levels of staff work together to accomplish some product or service. An example is a team comprised of an engineer, technician, and laborer. Foremost, the teacher-observer must adjust their conception of the work. The unit of observation is the job/work-unit. Because the job under study is situated by the team, its meaning is informed by the team. It is necessary to understand and tease out the everyday routines and relations--associated with the job/work-unit--that help the team function and understand how the job under study contributes. This may require an additional half-day on Day 4 to focus observation on everyday routines and relations that center around the team. Observation methods do not change.
When jobs primarily require workers to be highly mobile or itinerant, we make a different revision. Because highly mobile work tends to be heavily laden with relations, you will need to study job tasks on Days 1, 2, and 4, ensuring that you travel with different workers during that time. The travel time permits informal interviewing with frontline workers about tasks and work context.
Because of the relatively short time at worksites and because many jobs are technical and, in some cases, require licensure and certifications, you will not undertake participant-observation. Rather, you will depend on your observation and questioning skills and workers' knowledge. As a result, teacher-observers have to develop a comfortable relationship with workers over a short period of time.
We employ Spradley's framework for understanding social settings as an organizing guide for questions, observations, and fieldnotes (see Table 1). The social setting framework includes the following dimensions: space, actors, acts, activities, events, objects, goals, time, and feelings. Teacher-observers are asked to make holistic observations and then to document observations in a fieldnote using Spradley's framework as a guide. For new observers, this has proved to offer a workable prop for fieldwork chores.
Each day, teacher-observers will transform raw notes of job site observations into a complete fieldnote. This involves observers taking approximately one hour immediately following observations to flesh out the raw notes, using Spradley's framework. Within one to two days, each teacher-observer will produce a draft fieldnote and submit it to the mini-sabbatical project team for review and comment. We expect that each final fieldnote will average 15-20 pages for each half-day of observation.
Each teacher-observer will conduct formal face-to-face interviews with a department head and trainer. From their responses, you will have a fuller understanding of the industry, the company or firm, and the department. Important aspects may come to light that are not visibly prominent at the job level. A fuller understanding of work arms teachers with a rich perspective about authentic problems that can focus and shape high-quality classroom environments and lessons.
The interviews are formal in that you will employ an interview protocol comprised of a set of standard questions. The interview protocol is included in Section D at the end of the tutorial. Ask each question even if you believe you know the answer. Each interview will require approximately 20 minutes. During the interview, you will take raw notes which you will flesh out later. The previous day's informal interviewing, questioning, and note-taking will sharpen the skills required to accomplish face-to-face interviews. Your knowledge of the work itself should improve your ability to probe during questioning.
Table 1
Spradley's Nine Dimensions of a Social SettingSpace: Physical place or places Actor: The people involved Acts: Single actions that people do Activity: The set of related acts that people do Event: A set of related activities that people do Object: The physical things that are present Goal: Things people are trying to accomplish Time: Sequencing that takes place over time Feelings: Emotions felt and expressed Taken from Spradley, J. P. (1980). Participant observation.
New York: Holt, Rinehart, and Winston.
While accomplishing observations and interviews, the teacher-observer will need to look for artifacts that offer additional evidence of skills, tasks, and context operating in these natural settings. Ask permission from the department manager before removing documents or materials from the worksite.
The following artifacts are potentially helpful in designing authentic projects and investigations: organization charts; employee benefit package; annual reports; employee evaluation forms and procedure; company newsletters; syllabi for training programs; nonproprietary documentation of productivity; and job aids, reference materials, and procedures used by frontline workers to perform tasks.
The mini-sabbatical project staff will monitor each worksite on fieldwork days to answer questions about the application of Job Study procedures. On the evening of Day 1, we will debrief (by telephone) with each teacher about the following day's observation plan and note-taking. We refer questions about the work itself to the worksite and assigned mentor.
My name is ________________ and I am involved in a training project for high school teachers assigned to school-to-career programs. The project is located at (Training Site).
The purpose of this interview is for me to better understand work in the ____________ department, this company, and the (Industry Name). The interview is entirely voluntary. Answer only the questions you want to answer. Your responses are confidential. I will use them along with work observations to help develop curriculum for high school students.
The interview will take about 20 minutes. I will take notes throughout the interview.
Position Title: ___________________________________________________________
Length of Tenure at Company/Firm: __________________________________________
Tenure in Industry: _______________________________________________________
Tenure in Field: __________________________________________________________
| 1. | First, what are your primary job responsibilities? |
| 2. | What key issues are at the forefront of the (Name) industry (e.g., quality, efficiency, good government, environmental concerns, and so on)? |
| 2a. | Can you point me to reference materials that offer a good discussion of the issues (e.g., definitions, pro and con positions, and case studies)? |
| 3. | How is (Name of Company or Firm) coping with the key issues (e.g., improvement programs, restructuring, downsizing, new products and services, and community involvement)? |
| 4. | What is the mission of (Name of Company or Firm)? |
| 5. | What is the role of this department in light of the firm's mission? |
| 6. | I have been observing (Name of Job). For workers recruited as (Name of Job), what specific credentials, skills, or attitudes are considered crucial for solid job performance? |
| 7. | Alternatively, what would be "cause" for dismissing someone from this position? |
| 8. | The final questions focus on training. What company training does someone working as a (Name of Position) typically receive? |
| 9. | What are your training goals? |
| 10. | How do you evaluate training? |
| 11. | This concludes my questions. Is there anything you want to ask me? |
Thank you for allowing me to interview you.
During the past few years, concerns about the U.S. economy have prompted heated debates about vocational education and employment training programs. Experts have called for a variety of reforms as a result: changes in curriculum content, in teaching methods, and in methods of assessment. Reports on the inadequate skills of high school graduates, the rapidly changing demands of many employers, and the declining competitiveness of U.S. firms in the international marketplace have also stimulated a variety of proposals to change the organization and structure of employment preparation programs. Despite different approaches to reform, all sides seem to agree on the need for valid, reliable, and affordable methods for assessing students' skills.
This section is designed to help you choose the best assessment method(s) for your particular needs, by moving through the following steps:
Different types of assessment strategies are used for different purposes; your purposes should determine how you measure knowledge and skills.
For example:
One form of assessment is not necessarily better than another in all contexts. Rather, one form might be more appropriate than another, depending on your purpose, the knowledge and skills to be assessed, and quality and feasibility considerations.
Assessments designed to measure student learning in a course or program often become an integral component of the curriculum. Assessments with this purpose can be administered on-demand: they can be given often and graded quickly, and performance can be assessed cumulatively over longer periods of time.
Many teachers use assessments to measure individual student learning and progress. Some traditional strategies for this purpose include pop quizzes, end-of-chapter exams, and in-class essay exams. Alternative forms include portfolios, oral presentations, and senior projects. Assessments with this purpose can provide a teacher with insight into students' academic and technical progress and, if used over time, an impression of students' cognitive development as well. Assessments designed to measure student learning can also provide teachers with information about which instructional approaches work best, and where changes need to be made or additional attention focused.
An assessment used to certify student mastery of occupationally oriented material provides an effective means of signaling to employers that a student has a particular set of skills and knowledge. Since students generally seek employment (or a higher-level job) in an occupational area after being certified, employers frequently give input into the design and implementation of the system. In a true employee certification system, employers require certification for hiring and career advancement. In some industries, such as health, students cannot apply for particular jobs without the certification.
Assessments used to certify students may document general abilities or accumulated knowledge (e.g., tests for college admission) or specific ones (e.g., tests for professional licensing). Individuals who pass tests of specific, job-oriented skills often receive a certificate that can be used statewide, or even nationally.
Assessment is also used to provide information on the quality of the programs, schools, and districts that are providing education and training. This information may be used for monitoring progress in making program improvements, maintaining quality, making comparisons across different programs, or holding administrators of programs accountable. Trend data for programs, schools, or districts can also be used to monitor the performance of these entities over time.
Assessments used to provide program performance information often aggregate scores from individual performances to describe the achievement of particular groups (e.g., a graduating class or a whole school, or males and females separately). Assessments with this purpose are often administered infrequently, for example, annually or even less often.
Directions: List assessments that are used for one (or more) of the above purposes in your district or school, along with the purposes. What problems do you think you might encounter by attempting to accomplish more than one purpose with the same assessment?
This type of test is highly efficient for testing a student's knowledge of specific facts or skills and includes multiple-choice, true-false, or fill-in-the-blank questions. There are a limited number of predetermined "right" answers (sometimes only one); the student selects or produces a limited response to a stimulus or prompt. Open-ended questions are distinguished from the others in this category by the student having to generate a response rather than choosing from among ones presented. It costs less (in time and funds) to develop and administer a multiple-choice or open-ended item test than other forms of assessment.
These three types of written assessment require students to demonstrate their knowledge of a topic in writing, including a short answer or explanation or a long essay response. The stimulus can be printed material but it can also be an object, an event, or an experience. Written assessments include essays in response to a question, and problem- or scenario-based responses. This type of question may challenge students to think about issues and problems related to the industry that they are studying and often challenges students to integrate their knowledge of several disciplines. Usually scoring is more complex and time-consuming than for multiple-choice or open-ended assessments.
This type of assessment may consist of one or a set of multiple physical tasks such as changing the oil in a car engine or drafting a floor plan for a building. Performance tasks can be designed to test a student's specific abilities in a skill area, or his or her decision-making or problem-solving skills, or some combination of this type of skill. Performance tasks can be structured with one evaluator using a checklist of items and scoring criteria or by dividing observation among several different evaluators using a common scoring rubric.
Senior projects include at least three discrete activities to measure a student's achievement over the course of the senior year of high school (though the concept can be adapted for a different year or time period): (1) a research paper, (2) a project (the product is usually not written and is often an artifact of some kind: a videotape, a performance, or a physical model, for example), and (3) an oral presentation. Each component has its own criteria for evaluation. Evaluators are trained to properly assess a student's senior project. The time and cost required to evaluate senior projects is relatively higher than when a single written assessment or performance task is used.
A portfolio is a collection of student work covering multiple outcomes and activities. It is usually implemented with a focus on one of three purposes: (1) to improve curriculum and instruction, (2) to help students get jobs and to improve employability skills, or (3) to challenge students to take an active role in setting and meeting goals and in shaping their own tasks. The portfolio might represent work samples collected over one or more years. In some instances, a portfolio might include results from a standardized test or another written assessment instrument.
Directions: Read the following scenarios and answer the questions that follow:
| (1) | The
assistant superintendent in Greenwood School District is asked to design a
program improvement system for the district. The performance of each school not
only will be reported to the community, but will also be used to award bonuses
to teachers at the best schools.
What types of assessments would you use? Why? |
| (2) | Within
a particular state, biotechnology firms are interested in hiring large numbers
of skilled workers over the next few years. A group of CEOs contacts the
community college system to design a certification system for two job
categories that require an associate's degree.
What types of assessments would you recommend? Why? |
| (3) | Teachers
at Lifton High School have been disappointed in the scores of their juniors and
seniors on statewide standardized tests. They don't think the scores adequately
reflect students' achievement or abilities. These same teachers are also aware
that many students are disengaged from the high school curriculum by junior
year, which may be contributing to declining scores and low attendance rates.
These teachers would like to design assessment strategies that both engage the
students and help demonstrate their achievement.
What types of assessments would you recommend? Why? |
To ensure that an assessment strategy will provide accurate information, the technical quality of the measures should be considered. Three aspects of assessment quality are of special concern:
There are no perfect measuring tools, either in science, in the kitchen, or in education, so people who use tools to measure things need to know how much error there is likely to be in the information they receive. When we talk about the reliability of an assessment measure we mean to what degree the score on the measure (or on the test as a whole) is accurate. If, for example, a student took the same test again, would she or he get the same score? If students took a comparable test would a similar result be obtained?
On commercial tests, the reliability is usually around .80, which is considered high. This means that 80% of the test score reflects "true" performance and 20% reflects measurement error. High reliability comes partly from the fact that commercial tests obtain lots of separate bits of information about what students know; for example, students might answer 30 multiple-choice questions per half-hour.
Achieving high reliability with alternative assessment methods is more difficult. The usually longer and more complex responses supply fewer pieces of information about performance. Example: A teacher evaluating a portfolio is likely to review only a handful of student products, giving limited evidence. Moreover, judgment is required for scoring, inevitably bringing in a certain amount of subjective opinion. Subjective judgments may be reflected in inconsistencies between raters. Interrater reliability asks, Would two raters score the assessment the same way? Would the same rater, repeating the scoring session at a different time, score the assessment the same?
In selecting an assessment measure, we need to consider whether it would give the same result if repeated, how well its scores correlate with those of other assessments measuring comparable knowledge, and consistency across raters.
The validity of an assessment tells us whether it is measuring what we think it's measuring. If we want to know how well a student can write, a multiple-choice test of spelling and grammar may not be a valid indicator or how successfully he can write an essay himself, though it may indicate how well he can identify errors of this type in text.
There are several ways to establish or measure a test's validity. A panel of experts in the field can review the contents of the measure; performance on the test measure can be compared with actual performance on similar tasks in, for example, a work setting; or we can study the pattern of responses among several tasks measuring the same thing.
One of the primary motivations for adopting alternative assessments is to increase validity by making the assessment tasks more like the real-world activities the tests are designed to simulate. Because alternative assessments pose more "authentic" tasks, it is hoped that the assessment scores will more accurately reflect students' ability and knowledge in a given area.
One problem with interpreting the results of some types of alternative assessments, such as senior projects or portfolios, is that they are inherently nonstandardized. The content of each individual's submissions will be different, and the resources available to the student may vary. It is difficult to assign scores fairly to such different products, including taking into consideration factors like access to resources (such as computers or access to experts).
If students who otherwise have equal ability score differently on an assessment because of background knowledge or experience that is irrelevant to the assessed skill/knowledge, then the measure is unfair or "biased." Example: A task that assumes the student is familiar with different snow conditions may be biased against students who live in a climate where it never snows.
The fairness of an assessment is usually established by expert committees trained to analyze factors that might disadvantage or benefit particular groups of students. Many advocates believe that alternative assessments are more equitable to all groups because they assign more complete tasks and permit students to address the tasks in ways that are meaningful to them. However, all vocational educators selecting and constructing assessments need to be sensitive to the diverse backgrounds of their students.
Directions: By yourself or in a group, answer the following questions:
| (1) | Think of a test being used in your state or school but not in your classroom. What do you know about its reliability, validity, and fairness? What would you have to do to find out? |
| (2) | Think of a test being used in your classroom. What do you know about its reliability, validity, and fairness? What would you have to do to find out? |
Practical issues of cost and time required to administer and score, complexity, and acceptability are legitimate concerns in selecting from among alternative assessments. It should come as no surprise that selected-response tests are the most efficient users of time and budgets.
In general, alternative assessments are more expensive to develop, administer, and score than selected-response tests. Scoring is the greatest added expense of using alternative assessments. Multiple-choice tests can be scored quickly for only pennies per student. Because of their complexity, alternative assessments are time-consuming to score. Essays, for example, can cost several dollars per student to score. Often there are additional costs in training the people who will do the scoring.
Offsetting the additional costs of alternative assessments may be two benefits: (1) substantial staff development and (2) greater test validity. Teachers report that scoring alternative assessments improves their understanding of student learning, including their misconceptions and problems, and it is useful for instructional planning. Alternative assessments are also likely to provide more valid information about students' abilities to perform occupationally relevant tasks.
Alternative assessments place greater time demands on administrators, teachers, and students. Alternative assessments frequently require more class time to administer (which may cut into instructional time), and certainly require more time for scoring, which may reduce teacher planning time. On the positive side, teachers learn more about student performance by scoring this type of task. Moreover, when assessments are closely linked to classroom instructional activities, such as senior projects and portfolios, the distinction between assessment time and learning time is blurred, and the time problem may be less troublesome.
Alternative assessments are usually more complex than traditional tests. Students respond to more complicated test questions or situations that may be covering a broad range of course content; the method students use for responding is more elaborate; students may use manipulatives and may produce objects/artifacts in response to tasks; higher-order thinking skills are often required; and the scoring procedures are more complicated.
Making the arrangements to conduct an alternative assessment is also more complicated than passing out pencils and paper for a multiple-choice test. Training may be necessary to learn to administer or score alternative assessments (and sometimes to develop them as well), and additional equipment and facilities may be needed.
People familiar with traditional types of tests may be reluctant to implement alternative assessments or to accept alternative assessment results as credible. If the measures fail to meet reasonable technical standards or to address accepted curricular material, they may in fact be less credible. On the other hand, one of the advantages of alternative assessments is that employers and other stakeholders may give greater credibility to scores based on authentic performance tasks than to traditional test results.
Directions: Using the tests you thought of in the last section, answer the following questions:
| (1) | What do you know about the cost, complexity, and acceptability of the test used in your state or school? |
| (2) | What do you know about the cost, complexity, and acceptability of the test used in your classroom? |
The goal of the mini-sabbatical project is to help teachers make the kinds of curriculum and pedagogical changes implied by reforms to integrate vocational and academic education and to improve the school-to-work transition.
To make use of the occupation or industry as a context for learning, teachers need knowledge of work and work practice, a new model for classroom design and instruction, and a stress-free opportunity to learn and apply both.
Teachers participating in the mini-sabbatical will
The mini-sabbatical is designed for high school teachers in a variety of programs, whether they teach in academies, cooperative education, school-based enterprises, career focus schools, or other program types.
Teachers meet for six weeks (six hours a day, four days a week) beginning (date) and ending (date).
The mini-sabbatical will involve activities at three sites: (1) training site; (2) local worksite for observations; and (3) an area high school for two weeks teaching the newly developed curriculum.
Teacher participants will receive an honorarium of ($ amount). Student participants will receive a stipend of ($ amount) for two weeks of participation from (date) to (date).
To inquire further about this project, contact the Project Director at (telephone number).
| To: | Participating Agencies and Mentors |
| From: | (Name of Project Director) |
| Date: | (Date) |
| Re: | High School Teachers' Professional Development |
(Name of Training Firm) is conducting a training project for high school teachers who are involved in work and career programs. The goal of the project is to assist teachers in developing high-quality lessons for their students. Professional trainers will observe them teach the lessons they developed in an experimental summer school class.
During the week of (Date), teachers will spend several days at a worksite where they can gain exposure to authentic problems that workers deal with on actual jobs. Work observation provides a window into understanding authentic problems. Understanding the nature of an authentic problem and how it is solved provides the basis for designing the curriculum unit and student assessments.
Participating agencies will be able to influence teachers' training for developing curriculum tied to school-to-career programs. Though it is not the primary purpose of worksite observation, mentors will have an opportunity to learn more about bridging and connecting aspects of school-to-career programs.
Thank you very much for participating in the "High School Teachers' Professional Development Project." We think you will find the program challenging and rewarding, and we are very much looking forward to working with you for the next two weeks.
Remember, these are the days you will be needed at (Name of High School):
About your classes: You will be working on projects relating to transportation, health, or science. In some of the classes, you will have the chance to use computers and other types of technology. All of the classes will require teamwork and some kind of final presentation. Also, expect to do work outside of class!
About your pay: If you have completed the consent forms and survey, and if you attend all eight days, expect to get a check for $160 on or before August 15. If some kind of problem comes up--your forms are turned in late, or you can't complete all eight sessions--you will still be paid for all the days you do attend (at $20 per day), but there will be about a three- to four-week delay in processing your check.
Thanks very much for your help. Again, we are looking forward to getting to know you. If you or your parents have any questions, please do not hesitate to call (Staff Name) at (Telephone Number).
Sincerely,
(Name)
Project Director
(Name)
(Title)
(Firm)
(Address)
Dear (Name of Mentor):
We appreciate your willingness to share expertise and to provide assistance to (Name of Teacher). This project could not have accomplished its goals if people like yourself had not been willing to participate. Please extend our gratitude to your entire staff.
Observing work provides high school teachers with new perspectives and options for enriching curricula. (Name of Teacher) and seven other teachers are developing new curricula--influenced, in part, by work observations--and will teach it to a small class of students in a two-week experimental summer class which begins (Dates).
As part of this process, (Name of Teacher) will make several presentations to the other teachers and our project faculty before students arrive and then again at the close of the experimental summer school. The presentations will focus on the design of the new curriculum. We invite you to attend the presentations. Besides providing constructive feedback to (Name of Teacher), you might also have some general observations about teachers' curriculum development efforts and attempts to infuse real work aspects into curricula. We are videotaping the presentations and discussions that follow.
Date, time, and location information is as follows:
We hope that you are able to attend the presentations. If you are not, please accept our thanks for assisting (Name of Teacher).
Please call me if you have comments or questions about the project. I can be reached at (Name of Training Site and Telephone Number).
Sincerely,
Project Director
Appendix B-3 includes information provided to students participating in the mini-sabbatical: schedule; expectations, practice for journal writing, practice for activity reporting, and example activity checklist report (for student evaluation of daily lessons).
Welcome!
Thank you very much for participating in the "High School Teachers' Professional Development Project." We think you will find the program challenging and rewarding, and we are very much looking forward to working with you for the next two weeks.
Remember, these are the days you will be needed at (Name of High School):
About your classes: You will be working on projects relating to transportation, health, or science. In some of the classes, you will have the chance to use computers and other types of technology. All of the classes will require teamwork and some kind of final presentation. Also, expect to do work outside of class!
About your pay: If you have completed the consent forms and survey, and if you attend all eight days, expect to get a check for $160 on or before August 15. If some kind of problem comes up--your forms are turned in late, or you can't complete all eight sessions--you will still be paid for all the days you do attend (at $20 per day), but there will be about a three- to four-week delay in processing your check.
Thanks very much for your help. Again, we are looking forward to getting to know you. If you or your parents have any questions, please do not hesitate to call (Staff Name) at (Telephone Number).
Sincerely,
(Name)
Project Director
High School Teachers' Professional Development Study
Experimental
Summer School Schedule
(Date)
(High School) Campus
| (Date) | |||
| 9:30 a.m. | Opening Assembly | Project Director | Multipurpose Room |
| 10:30 a.m. | Learn and practice writing reports | Staff | Multipurpose Room |
| 11:30 a.m. | Class | Assigned Teacher: ________________ |
Assigned Room: ________________ |
| 12:30 p.m. | Dismissal | ||
| (Date) - (Date) | |||
| 9:30 a.m. | Class begins | Assigned Teacher | Assigned Room |
| 11:40 a.m. | Complete reports | Staff | Multipurpose Room |
| 12:45 p.m. | Dismissal | ||
| (Date) | |||
| 9:30 a.m. | Teachers' final presentations | Project Director | Room TBA |
| 11:30 a.m. | Feast | ||
| 12:30 p.m. | Dismissal |
Expectation: Each student will write a one-page journal entry and complete an activity checklist report each class day.
| Q: | Why are the daily journals and activity checklist reports important to this study? |
| A: | The study is intended to help teachers develop some new classroom methods and student feedback is needed to help us determine how the new methods work for students. |
| Q: | What happens to the journal pages and activity checklist reports? |
| A: | The study project leaders will hold on to them. We will not share them with the assigned teachers. They become part of the data used to develop a final version of the mini-sabbatical. The privacy of student authors and confidentiality will be maintained. |
| Discussion: | For those that have journalled before, what is the benefit of journalling? What if you can't think of anything to write? |
| Practice: | For the subject "I am a student," respond to the following prompts: |
The activity checklist is a three-part report. The following aspects are covered:
STUDENT CHECKLIST REPORT
| Name: __________________________ | Student ID Number: _______________ |
| Teacher: ________________________ | Teacher ID Number: _______________ |
| Date: ___________________________ | Project Day: ______________________ |
| 1. | Topic: What was today's lesson about? | |||
| 2. | Content: What did you do during the lesson? (Check ALL that apply!) | |||
| ____ | Read (books, encyclopedias, magazine articles, handouts) | |||
| ____ | Watched video, movie, TV program | |||
| ____ | Listened to recording (tape, CD) | |||
| ____ | Looked up information from a CD-ROM encyclopedia or the Internet | |||
| ____ | Got information from a spreadsheet or ledger | |||
| ____ | Got information over the phone or by interviewing an expert | |||
| ____ | Looked at drawings, layout plans, photographs, or model | |||
| ____ | Listened to the teacher present information, or demonstrate a procedure or skill | |||
| ____ | Listened to a student present information, or demonstrate a procedure or skill | |||
| ____ | Listened to a guest present information, or demonstrate a procedure or skill | |||
| ____ | Conducted a lab experiment | |||
| ____ | Took a survey | |||
| ____ | Conducted a survey | |||
| ____ | Summarized information from an experiment or survey | |||
| ____ | Took notes | |||
| ____ | Worked with a group to solve a problem | |||
| ____ | Worked by yourself to solve a problem | |||
| ____ | Created a piece of writing | |||
| ____ | Prepared or gave an oral presentation | |||
| ____ | Made comments/gave feedback on a student's presentation or writing | |||
| ____ | Got comments/feedback on your presentation or writing | |||
| ____ | Made a video or audio recording | |||
| ____ | Created a multimedia presentation on the computer (including text and visuals) | |||
| ____ | Created a drawing, layout, or photograph | |||
| ____ | Created a model or other object | |||
| ____ | Created or entered information into a spreadsheet or ledger | |||
| ____ | Took a test or quiz | |||
| ____ | Reviewed a homework assignment | |||
| ____ | Took part in a team-building activity or discussion of group roles | |||
| ____ | Led a group or recorded for a group | |||
| ____ | Other: ______________________________________________________ | |||
| 3. | About what percent of today's class did you spend working: | |||
| * | by yourself? | ______ % | ||
| * | in a group? | ______ % | ||
| * | with the whole class? | ______ % | ||
| (Total) | __100_ % | |||