Evaluation Findings

Before reviewing findings related to each goal, we note that teacher ratings on the cumulative evaluation form were very positive overall: average rating ranged from 3.6 to 4.9 (33 items covering activities in four areas). One activity received a rating of "2" from one teacher; otherwise, all items were rated 3 or higher. Uniformly, teachers had very positive comments about the experience as a whole. When asked if they had any suggestions for improving the mini-sabbatical experience, one experienced teacher wrote, "No! The professional development mini-sabbatical was very interesting, exciting, well-prepared, interactive, and challenging!"

In the following sections, we describe the set of mini-sabbatical activities related to each goal, then discuss findings related to achieving each.

Goal 1: Increase Teacher Knowledge of Work Practice

For most teachers, the activities designed to increase their knowledge of the world of work, as related to their specific discipline, were very successful and meaningful. Again, the goal of this phase of the mini-sabbatical was for teachers to understand workplaces, not merely to visit them. At the end of the first week, teachers were introduced to the skills they needed to perform, analyze, and document worksite observations. Presentations by mini-sabbatical trainers addressed several topics: (1) authentic practice, work context, and the rationale for worksite observations; (2) understanding work from workers' perspectives; (3) techniques for observing and documenting work; (4) types of tasks suitable for the design of high-quality learning experiences; and (5) the logistics of the workplace observation scheduled for Week 2 (e.g., assigned mentor, schedule, and so on). Teachers had an opportunity to practice observation and documentation techniques by shadowing a RAND employee. Work observations included an electrician, a computer trouble-shooter and repairer, and a public relations officer.

Teachers spent the following week at assigned workplaces to observe work practice, take fieldnotes, and interview their mentor. We attempted to match teachers to worksites and mentors based on the teachers' disciplines, their school programs' industry focus, and the teachers' initial ideas about the curriculum unit that they were going to develop. Five teachers working in transportation career academies were assigned to various departments at the Los Angeles County Metropolitan Transportation Authority (LACMTA), which is a business partner with the school district that houses these academies. Others conducted worksite observations at a local university and two public agencies (see Table 3).

During the observation phase, mini-sabbatical staff reviewed and commented on teachers' fieldnotes to ensure that teachers were focusing their observations to capture information about work organization, workplace skills, and other aspects of work life that could inform their curriculum development projects. Mini-sabbatical staff were also "on call" in case teachers had questions or encountered problems during their worksite visits.

Except for the transportation career academy English teacher, whose original mentor assignment did not work out as expected,[8] the teachers learned a great deal from their worksite observation and interviews. Teacher journals and fieldnotes highlight important themes such as understanding the difference between domain-specific and generic skills and identifying authentic work problems that can animate the design of project-based work in the classroom. The following journal entries illustrate teacher experiences in their own words:[9]

I saw a brainstorming session in which each person, knowing what they bring to the table, created a powerful example of people working together to solve a problem--to get out a simple message, the uniqueness of the cancer center. . . . If I can create curriculum with structures to allow students to bring to the table their experience and knowledge to work through a problem based on literature or media, it will feel great to transfer the creative chaos, with reason never far behind, to the classroom. (Teacher 6, Medical Center Marketing Department) After having observed for two days, I finally am beginning to feel like I can do this. I am learning a lot of additional information about laboratory procedures. I see how the plant works. This is a rich source for planning biology labs. (Teacher 7, Standards and Testing Laboratory) Some of the generic skills that I think would be transferable to the curriculum I'm writing . . . (1) how to communicate effectively in a small group setting and make so that everyone has a job to do according to the level of expertise; (2) how can the project be managed effectively so that a complete product is the endpoint; (3) how can motivational techniques be utilized to keep student on task over a large period of time? (4) what kind of goal setting techniques can be utilized? (Teacher 5, DWP Executive Offices)

During debriefing and discussion sessions, the teachers shared observations about their worksites and implications for their curriculum and teaching. After even two days of observation, several important themes emerged from their discussion which suggest that teachers were learning valuable lessons and new information about work practices. For example, several teachers discussed the importance of interpersonal relations at work, and the need to work with different types of people to build consensus. They noted differences in types of workplace communications (e.g., informal hallway conversations versus a formal meeting for sharing information) and the importance of having good communication skills. They also discussed teamwork and interdependencies among jobs and departments, particularly for projects or tasks that require different types of expertise (e.g., subway station design and construction, multimedia presentations). They noted differences in management style, in particular, ways to motivate staff and organize projects. One teacher, for example, considered how techniques used to motivate staff in the workplace might be applied to motivating students in the classroom. In discussing their individual worksites, teachers agreed that workplaces are not always "easy places."

From these and other insights gained during their worksite observations, they began to identify skills and knowledge to incorporate into their curriculum plans. In addition, they also began to think differently about the workplace learning experiences that their school programs provided for students. Several teachers admitted that they had never thought much about these work placements because they were organized by a program coordinator or because they were not required to interact with employer sites.[10] Their own experiences prompted new considerations: Are we doing a good enough job of preparing students to enter a real-work environment? Should we give students specific tasks to accomplish at their work-based learning sites?

Interestingly, even teachers with previous related work experience found the fieldwork valuable. For example, one teacher with experience working as an architectural draftsman reported that his critical moment or experience in the mini-sabbatical was related to what he learned in the workplace: "Using problems from MTA, as opposed to standard educational texts, creates the classroom into an office."

We noted some differences in the way academic and vocational teachers initially focused their observations. Vocational teachers tended to concentrate initially on the domain or technology-related skills used at work. This perhaps reflected the traditional role of vocational educators to prepare students for jobs, when the teachers are responsible for making sure students acquire job-related technical skills. The training and feedback in the mini-sabbatical was essential for helping vocational teachers broaden their view of work and workplaces to include social aspects of working or work problems that incorporate technical skills. Further evidence that the worksite observation activities enhanced teachers' integrated curriculum development is presented later in the text.

Goal 2: Create High-Quality, Integrated Curricula

Throughout their worksite observations, teachers began the process of thinking about how to incorporate aspects of work practice into their curriculum plans. Curriculum development activities (Weeks 3 and 4) first included an exercise to help teachers move from worksite observation to instructional design--that is, from job tasks to authentic problems. Mini-sabbatical teacher-trainers led a discussion about authentic practice, then asked teachers to discuss and write a summary of their own job study.

Their summary addressed several dimensions, including skills, tasks, and work context for the job; authentic problems; categories and examples of instructional goals that address authentic problems; and aspects of a hypothetical classroom environment. Teachers read and discussed alternative approaches to developing integrated curricula, and reviewed the CTW model. Teachers were asked to build their new curricula around a project or investigation based on authentic practice and solving authentic problems. We provided an instructional design template for teachers to specify several elements of their design: summary of student product, instructional goals (e.g., generic, domain, attitudes, or dispositions), design (e.g., culture of practice, teacher role, assessment, classroom set-up), teaching methods, resources required, and organizational supports (e.g., coaching by mini-sabbatical trainers or peers, preparation time). In subsequent sessions, teachers had opportunities to modify this "baseline" design and provide a rationale for any changes they made.

One way to assess teachers' progress in curriculum development is to compare the types of lessons and units they initially proposed prior to being selected as mini-sabbatical participants with the projects and topics they began to refine during Week 3. On the pre-course survey, we asked teachers to submit a previously taught curriculum unit that they planned to refine during the mini-sabbatical. If they did not have a particular unit in mind, but planned to create a new unit for an existing course, we asked for the course summary or syllabus. As a third alternative, teachers could submit a curriculum unit that they believed motivated students' effort. Teachers also answered a series of detailed questions about the curriculum. This comparison reveals some significant changes (see Table 4).

One clear difference is the emphasis on group work over individual learning assignments. Six teachers began with individual student projects or assignments, but all designed team-based projects for their final curriculum.

Although some teachers initially proposed projects, their final projects were much more "authentic" in their connection to real-work settings. A CAD/drafting teacher, for example, initially proposed to develop a project that he already conducted in his class--to build a popsicle bridge as a way to illustrate principles in drafting, math, and science. The requirements for the bridge project came from a contest held by the local chapter of the American Society of Engineers, where schools could actually send teams of students to a bridge-building competition. For this teacher's final project, students designed a bus parking lot on a real site, given a set of specifications drawn from actual design requirements by a county facilities engineering department (where he had done his work observation). Students worked in teams as design engineers or architectural drafters and produced an actual plan. This project supported many of the same skills as the popsicle stick bridge (e.g., drafting, math, problem solving), but also incorporated other work-related, generic (e.g., teamwork, communication, and presentation) and technical skills (understanding spatial relationships, two-dimensional area planning). In addition, the students' final product was not a toy model, but an actual plan of the type that working engineers and architects produce.

As the previous example also illustrates, another significant change was the integration of academic skills, generic skills, and specific competencies needed to carry out a project. Although their initial projects were often interdisciplinary or explicitly connected to other classes in the school program (e.g., the English teacher's assignment for students to write reports about work conducted in their technology class), they did not typically emphasize or articulate work-related skills. When teachers came to the mini-sabbatical, curriculum integration typically meant "interdisciplinary." By the end of the mini-sabbatical, however, they learned to incorporate other aspects of integration into their lesson planning and instructional goals, namely the connection between school and work.

Teachers were also inventive in defining their roles and in creating a culture of practice in the classroom. The biology teacher, for example, became a laboratory supervisor (who occasionally adopted the role of lab assistant) to her students--the "testing laboratory" scientists. On the first day of class, she handed out a memo to the "scientists" at the "Wilson County General Services Division, Testing Laboratory" that outlined training they would receive in the lab and what their work duties would entail. The teacher/lab supervisor wore a white lab coat and fitted the classroom with test tubes, thermometers, chemicals, and other materials needed for water testing.

Goal 3: Adopt Teaching Roles To Support Authentic Learning

Teachers were introduced to the CTW model during the first week of the mini-sabbatical through a set of briefings, readings, and journal writing exercises. Concepts were reinforced in Week 3, when teachers began to develop their curriculum. Teachers had opportunities to practice new teaching methods during Weeks 5 and 6, when they taught their curriculum units. Teachers received coaching from mini-sabbatical staff and benefited from videotape feedback and group discussion.

Teacher evaluations indicate that the curriculum materials and processes were useful for developing teachers' understanding of the CTW model. Six teachers' journal entries during the first week emphasized developing teaching goals, re-defining teacher and student roles, thinking of students as responsible learners and problems solvers, and working collaboratively with other teachers on curriculum and practice issues. One experienced teacher remarked, "I'm beginning to realize that after taking time to meditate on what makes `classrooms that work' that I have much to learn."

Overall, while teachers were generally familiar with the concepts of student-centered learning, cooperative learning, and the like, they had not been introduced to a comprehensive model that outlined specific teaching practices or design principles for implementing such concepts. Nor had teachers had an opportunity to participate in professional development that allowed them to systematically explore and reflect on the implications of the model for practice. As teachers reflected on the CTW model and what implementation of CTW concepts actually means for their practice, they began to develop useful insights. In Week 4, just before teachers went into the classroom, an experienced teacher wrote,

Last night I read "A Tale of Two Classrooms" and realized that experience as a teacher does not necessarily mean an effective teacher. . . . I am challenged to ask what skills should I be teaching? What is the classroom design? What teaching technique will I use?

As discussed above, the CTW model defines several specific techniques that teachers should adopt to enhance student-centered learning such as coaching, scaffolding, and fading. Adopting these techniques requires fairly significant changes on the part of teachers because they must give more responsibility to students for their own learning and not always take center stage. While teachers supported such pedagogical techniques in principle, they found it much harder to put them into practice. Some indicated that changing this aspect of their teaching practice was the most difficult and challenging part of the mini-sabbatical. In particular, teachers struggled with relinquishing "power" and control, and trusting the student groups to succeed with less intervention on their part. Teachers wrote about and openly discussed problems associated with giving up power and control:

Very difficult for me to relinquish control. I feel that I need to be at every phase of the project, making sure that they are doing it right. (Teacher 5, Week 5) I have felt strained over the changes necessary to becoming more conscious of my teacher role in becoming a "coach" and letting go of my centered ego. (Teacher 6, Week 5) My teaching style of old is still in evidence. I think it stems from my reluctance to let them or trust them to explore on their own. I need to find my niche in the class in order to fulfill my role as a facilitator. It's an issue of control. (Teacher 8, Week 5) I must think and rethink my role as a teacher because I like being the person that all the students run to for all of the answers. I have a hard time backing off and just letting the students learn on their own. (Teacher 2, Week 5) Old habits are hard to break and it's very hard for me to turn over the control of the class over to the class. But, I also understand that if I want them to be responsible for their own learning, I have to turn over that responsibility. (Teacher 8, Week 5)

Teachers also mentioned their successes in changing practice, despite the difficulties they initially experienced:

Today I felt more comfortable in the classroom. I trusted the class to complete the work. They did not disappoint me. (Teacher 4, Week 5) I'm pleased that the process is working! I felt somewhat insecure because they didn't "NEED" me. (Teacher 4, Week 5) I started to have them do all of the work in the classroom and I would transfer their work to the computer at home and return with the finished product. However, I decided that they would benefit more from doing the entire project themselves. (Teacher 7, Week 6) As a teacher who loves to be very involved in the process, it was very difficult for me not to lead the discussion; but I stepped away and behind the group. (Teacher 8, Week 6) I am continuing to keep a low profile and am encouraging them to take responsibility for their work. I think it has a motivating factor because they can take ownership of the project and not feel that they are doing something for the teacher. (Teacher 3, Week 5)

Teacher journals also illustrate the opportunistic aspects of teaching, where classroom events present a situation that a good teacher can use to advance his or her instructional goals (Stasz, McArthur, Lewis, & Ramsey, 1990). One English teacher, for example, reported an incident where a student objected to her starting the class five minutes ahead of schedule because students were only being paid for a certain amount of time. She used this objection to talk about workplace attitudes:

I explained that I was aware of the time, but I needed to bring a few things to the attention of the group before I forgot. . . . I took a few moments to explain again about the importance of the employee's attitude and work ethic. I cited the example of a manager seeking to promote someone to a new position. The attitude of the worker comes into play when a manager must justify a decision to promote one employee over another. Other students in the class immediately began to shake their heads affirmatively to signal their agreement with me. . . . Then I moved the discussion to the business at hand. (Teacher 4)

A key tool in facilitating changes in teachers' classroom roles was the use of videotapes. Set up on the model of movie production "dailies," where participants can view and discuss the results of a day's work, mini-sabbatical teacher-trainers videotaped portions of the morning classes, then led debriefing sessions in the afternoons. On one afternoon, a teacher-trainer turned the sound off and simply had teachers observe the physical organization of the class and the extent to which students were actively engaged in a learning task versus simply listening to the teacher talk. These discussions allowed teachers to see themselves and their colleagues, and to measure their own progress toward acquiring the teaching techniques outlined in the CTW model.

Goal 4: Develop Alternative Assessments

Of all the mini-sabbatical goals, this one perhaps proved most challenging for teachers. During Week 3, teachers participated in a presentation and discussion of alternative assessment covering purpose; types of assessment; and issues of a variety of topics, including reliability, validity, and feasibility. They also worked on some exercises which aimed to clarify these and other assessment-related issues. Even experienced teachers had difficulty thinking about how to assess students' performance in ways that aligned with all of their instructional goals:

This week has been very informative for me. I have had the opportunity to actually find out what assessment is. The presentation that I received went into extreme detail, which really helped. However, I did not have any experience in assessing popsicle stick bridges or jacks. Assessment will be a challenge when determining the preferred outcomes from students. (Teacher 1) The assessment piece will probably be the most challenging. The documents and readings on assessment were quite helpful--but I guess I need some "hand holding" through this because my usual assessment practices include tests that measure nothing about learning. (Teacher 8)

As the following quotations suggest, teachers did gain some insights and understanding about assessments during the mini-sabbatical:

I will need to provide the assessment criteria to my students so they know what to do and I know how to grade. (Teacher 3) A deeper realization is to connect closer my assessment criteria with the work students are actually doing. Tied to this is the idea that students need more precision--as to my expectations and the reasons for my assessment criteria and curriculum goals--make my thinking plain and vocal. Let them see and hear how I think about thinking. (Teacher 6)

Some issues were problematic for most of the teachers. The group had a lengthy discussion, for example, on how to assess the quality of students' designs, as several projects included a design element (e.g., monument bridge, parking lot plan). What criteria should be used to assess the designs and how should criteria be presented to students? The following journal excerpt discusses this issue as one of developing scoring "rubrics," and the trade-off between validity (measuring what the students are taught) and feasibility (teachers' time is limited):

My problem with designing a rubric is that I want it to be fair to the learning of the students, yet I don't want it to have to take forever to make. If it does, I don't know how to convince my staff to do it regularly. (Teacher 8)

Teachers also discussed whether to assign individual or team grades and what criteria to use for team assessment. They also discussed the constraints of assigning team or project grades, since each student must have an individual grade in a class. Since these were experimental classes, in which grades had little meaning, teachers did not actually assign grades to their students. The discussion about assessment was still important, however, because teachers would need to grapple with these issues when they implement their curriculum units in their actual classrooms.

In the final analysis, the mini-sabbatical was successful in getting teachers to think explicitly about assessments, but they were unable to really develop formal assessment procedures. Even so, teacher journals indicate that the teachers constantly evaluated the learning process as students worked on their projects. This assessment was not formal, or explicit, in the sense that teachers decided beforehand to track particular student behaviors or look for certain signs of progress. Rather, they seemed to monitor what students were doing, and then recorded what they saw and heard. In addition, they recorded their responses to student work, such as the type of feedback they provided or the suggestions they advanced to assist the groups. Thus, even though teachers did not always think formally about assessment, nor consider technical aspects such as validity or reliability, they nonetheless tracked student progress and assessed student performance.

[9] We selected illustrative teacher quotations for this report and did not attempt to represent all the teachers in each reporting instance. All teachers, however, submitted journals and completed the evaluation form.

[10] At the Transportation Career Academy Program, student internships occurred over the summer. At the Medical Magnet High School, teacher involvement primarily consisted of visiting worksites to check student attendance. Teachers did this on a rotating basis.