Grubb, W. N., Kalman, J., Castellano, M., Brown, C., & Bradby, D. (1991). Readin', writin' and 'rithmetic one more time: The role of remediation in vocational education and job training (MDS-309). Berkeley: National Center for Research in Vocational Education, University of California.

DIRECTIONS FOR FUTURE POLICY

• Reforms for Now: Coordination, Effectiveness, and Pedagogy
• Reforms for the Future: The Purposes of Remedial Education

Every director of a remedial program whom we interviewed forecast an increasing demand for remedial education. They expect a continued expansion of developmental education in community colleges and continued pressure on the adult education system. The JTPA program has begun to emphasize longer-term training for the most disadvantaged individuals, so JTPA administrators also predict the need for more basic skills training. Most welfare-to-work programs have found greater academic deficiencies than they predicted, so welfare administrators also see a need for more remediation than the system is currently providing. Most forecast a continuing increase in the students leaving high school who are unprepared for the labor force. The "new demographics"--the increase in children born in poverty, the relative increase in minority children, and the continued immigration of people from the non-English-speaking counties of Asia and Latin America, all increasing the population in need of adult education--is now consistently cited as another source of demand. The furor over rates of illiteracy also suggests that there is a great deal of hidden or latent demand for adult education and the discovery of new forms of illiteracy--now with debates over "workplace literacy"--bolsters that opinion. If administrators of remedial programs were entrepreneurs in a conventional market, they would be ecstatic about the booming demand. As it is, they face increasing demand with dwindling resources--and the future looks pretty dismal.

Clearly, then, the direction of remedial education is far from academic. The current system of programs spends large sums and enrolls large numbers of individuals to achieve results which are uncertain at best, and nonexistent at worst. Continuing to expand the current system without substantial changes in the way the system operates seems foolhardy. In this section, then, we propose some directions that policy might take, concentrating on federal issues. Since federal policy often shapes state practice, especially in the areas of vocational education, job training, and welfare, we expect that changes in the federal role will influence state policy as well. If nothing else, the federal government can play a role in research, evaluation, and the analysis of alternative directions for remedial education that can influence state and local developments.[62]

In thinking about future directions, it is helpful to consider two different kinds of issues. The first includes those that can be addressed now--with current institutions, practices, and levels of funding. The second includes those problems that are much larger in scope, problems that call for fundamental changes in the institutions providing K-12, adult, and remedial education and substantial increases in funding that may seem unattainable for now. Unfortunately, the issues we have included in the second set include the most basic questions of purpose, so the way we have posed issues may seem backwards. However, we think it important to begin the process of re-examining and reforming the current system of remediation rather than remaining paralyzed by the impossibility of the task; so we begin with some relatively simple steps that could still provide the foundation for more substantial changes later.

Reforms for Now: Coordination, Effectiveness, and Pedagogy

Given the existing resources and institutions that provide remedial education, there are three kinds of reform that can be undertaken without substantial increases in resources or institutional reconstruction. The first involves coordination. In one sense, we see a great deal of coordination in the existing system of remediation. Even though most communities have a large number of basic skill providers, there is little outright duplication, partly because the demand is so great and the needs of potential students are so varied that every program can find its own niche. There is also a great deal of referral among programs, especially from federally funded programs with constrained resources--JTPA and welfare programs--to the largely state-supported programs in adult education and community colleges, which are more likely to have open-ended funding based on attendance. But rather than being a source of comfort, the current patterns of coordination are worrisome because of the lack of mechanisms to follow individuals among programs; from the viewpoint of individuals, the system probably appears random, unplanned, and poorly articulated. That is, the coordination through referral that now takes place does not ensure that individuals receive the services they need. Until there are better ways of tracking students--to see whether they do enroll in a program to which they are referred, complete the program, and then return for job skills training--the current referral system must be considered a black hole into which individuals disappear, never to return to the education and training system.

We are also concerned with the inattention to the quality and effectiveness of the programs to which individuals are referred. We found almost no instances in which programs have any information about even the crudest measures of quality (such as completion rates) of the remedial programs to which they refer individuals. Indeed, even those using their own resources for remediation--like many JTPA and welfare programs--have consistently failed to articulate any policy about basic skills instruction or any concept of what such programs should include. (By and large, administrators seem relieved to find any programs to accept their low-performing students. Because they see themselves as job trainers, vocational educators, or case workers, and not as adult educators, they don't want to establish their own programs.) We are particularly concerned with the most common practice of referring individuals to the adult education system: The extraordinarily high rates of noncompletion in ABE, the lack of evidence about outcomes, and the fact that most ABE programs follow conventional skills and drills approaches organized around completing the GED indicate that the most extensive referrals are being made to the least effective programs. In contrast, community college programs are in many cases more extensive, establish more meaningful goals such as entry into college courses or vocational education, and are much more likely to experiment with eclectic approaches to teaching.

Our first recommendation, then, is that coordination between remedial education, on the one hand, and vocational education and job training, on the other, needs to be more carefully considered. Referrals without tracking mechanisms and referrals to programs of unknown effectiveness are likely to be ways of diverting individuals from access to vocational education and job training, not ways of coordinating the different resources of a community. As a result, programs providing job training and vocational education--JTPA, welfare programs, community colleges, technical institutes, and area vocational schools--should as a matter of federal or state policy establish tracking mechanisms for individuals they refer elsewhere, establish policies and goals for the remediation to which they refer individuals, and consider more carefully whether established programs meet the goals set for remediation. To be sure, it may be possible to meet these goals in various ways: A community could establish a centralized clearinghouse, for example,[63] and those areas where a community college provides almost all adult education and vocational education[64] will find it easier to coordinate remediation as well. But however it is accomplished, the intent of our first recommendation is to improve the current situation where referral takes place with little knowledge of the consequences.

Obviously, the intent of the first recommendation is to require programs to refer individuals only to adult education programs that are effective, and this raises the second substantial problem: the task of examining more thoroughly the effectiveness of remedial programs. The current situation--in which some programs don't even have enrollment information, most don't keep data on completion rates, and almost none have even the crudest measures of outcomes--is one in which there is no way of improving the system because there is no information about which components work well and which are ineffective. We recommend, therefore, that resources for evaluation be increased. In particular, because states are often too small to develop their own evaluation programs, we recommend that federal agencies--particularly the Departments of Education, Labor, and Health and Human Services, with their responsibilities for the federal ABE program, vocational education, JTPA, and JOBS--take more seriously the task of evaluating remedial and adult education. Such evaluation should take two forms: the support of sophisticated evaluations, including those using randomized assignment and multiple approaches to remediation, in designs that would address which approaches are most effective for which individuals; and the development of smaller-scale, less sophisticated evaluation mechanisms that can help individual programs judge whether their students and clients are making adequate progress.

The lack of evaluation in developmental and remedial education is one of the most common complaints over the past twenty years (Roueche, 1968; Roueche & Snow, 1977; J. E. Roueche, 1983), and it is not clear why our recommendation for improved evaluation should make any more difference than the previous appeals. However, several developments make the problem of evaluation even more urgent now than it has been in the past. One is simply that the magnitude of the problem has continued to increase, and the funds being spent on remediation are clearly enormous (though unknown with any precision)--and the underlying causes show no signs of abating. Second, while the majority of funding for remediation comes from state governments, the role of the federal government has clearly increased with the resources in the ABE program, those in the Vocational Education Act used for developmental education, the shift in the priorities of JTPA toward individuals with more skill deficiencies, the establishment of JOBS, and the new workplace literacy programs funded by the Department of Education. The expansion of federal funding suggests a new responsibility in evaluation to make sure these funds and the state resources they leverage are not badly spent.

To be sure, the recommendation to improve evaluation is itself fraught with conceptual and technical problems. The technical difficulties include the usual issues that arise in devising evaluations of programs for which noncompletion is high, control groups are difficult to establish, selection effects (e.g., the tendency for more motivated individuals to enter programs) are powerful, and long-term results may differ substantially from short-term gains. But these pale before an obvious conceptual hurdle: Evaluation requires defining and measuring the outcomes of a program, and defining appropriate outcomes involves the deepest debates about what literacy entails and what this country wants of remedial education. Outcomes could be measured with conventional test scores or attainment of the GED, for example; but this outcome defines literacy and numeracy in terms of narrowly defined skills and reinforces pedagogies based on skills and drills. Those advocating whole language approaches have developed quite different forms of assessment, using open-ended questions, conversations between teachers and students, portfolios, self-evaluation, and other techniques still in early stages of development[65]--but these methods cannot be coded and quantified as standardized tests can. Outcomes could be measured by the success of individuals in subsequent or concurrent job training; but students in remedial programs may have different goals or may have trouble in job training for reasons that have little to do with their command of basic skills. Similarly, it is tempting to measure effectiveness in terms of subsequent employment, earnings, welfare dependency, mobility up career ladders, employer satisfaction, and other measures of labor market success, but doing so assumes that the economic consequences of programs are more important than other effects, and variations in labor markets create problems in comparing programs in different areas. Those who have discussed literacy have elaborated what its purposes are, developing conceptions of "multiple literacies" in different contexts that further complicate the measurement of outcomes.

Indeed, the most difficult questions we raise in the next section are those about purpose, and it is logically impossible to carry out evaluations without resolving these questions--at least partially. However, the issue of evaluation is much too important for us to wait until we as a nation have decided what we want from remedial programs. The only solution to the conundrum of how to evaluate programs whose purposes we have not adequately defined is to begin developing evaluations with many outcome measures, including those from different ideological positions and pedagogical traditions. Only with such multi-dimensional evaluations will it be possible to move toward a better understanding of what different programs can and cannot accomplish.

Finally, the issue of evaluation raises again the question of pedagogy. Our third recommendation is that those with influence over remedial programs--policymakers, administrators, teachers, researchers, evaluators--need to confront the issue of appropriate pedagogy. As we argued in Section Three, "The Nature of Effective Programs: The Conventions and the Structure of Skills and Drills," the usual evaluations--which compare a particular program's members to a control group that receives no remediation to determine whether the program has any effect (usually on test scores)--pose the question in a way that the only decision can be to continue or abolish the program. If evaluations are to be useful for improvement, however, they must compare alternative approaches and pedagogies--different student-teacher ratios, different intensities, different mixes of individual and group instruction, different mixes of computer-based components, alternatives to skills and drills, different uses of functional context training, and different kinds of eclectic approaches. Improvement of remedial programs and the evaluation that would help improvement need to consider pedagogy more seriously than has been true in the past.

Here, too, there is an obvious barrier. Policymakers, administrators, and most researchers contributing to national policy debates typically don't discuss pedagogical issues. They have different training than teachers; they have different concerns, often far removed from the classroom. The past eight years of debate over education has failed even to raise the question of whether teaching methods are appropriate.[66] Above all, teaching methods have always been nearly impervious to the control of administrators and policy-makers. If careful evaluation did confirm, for example, that methods based on skills and drills are ineffective for some students and for certain educational goals, then it would be difficult to force programs to adopt different methods by using the conventional regulatory mechanisms of policy. A much more elaborate process of revising teacher training programs, changing certification requirements, eliminating the now familiar assessment mechanisms (standardized tests, for example) that encourage skills and drills, and promoting good practice by demonstration and example would be necessary.

But, unfamiliar as it is, we see no way to evade the issue of pedagogy. The prima facie case against the skills and drills methods that dominate remedial education is too powerful to ignore. The high dropout rates, the many reports of student dislike and boredom, the lack of any substantial evidence that programs work all outweigh the occasional hopeful stories about individuals who have succeeded in the system. The logic of using a method which has previously failed many adults in remedial programs--individuals who have not learned basic academic competencies through eight to twelve years of conventional schooling--is incomprehensible. The ambitions of most skills and drills programs--which hope only to get their students through arithmetic operations with fractions and decimals, through reading for simple comprehension only, through writing a simple paragraph, through a GED exam with uncertain consequences--are painfully limited, especially in a period when national pundits and business interests are calling for "higher-order" skills.

These three issues--coordination, effectiveness, and pedagogy--are tightly linked. The appropriate coordination among vocational education, job training, and remedial education programs cannot take place without evidence about effectiveness. Better information about effectiveness cannot be developed without considering alternative pedagogies, and the actions necessary to improve programs will need to consider how to put different teaching methods into place.

Reforms for the Future: The Purposes of Remedial Education

A great deal of improvement in remedial and development education programs is possible without settling the overarching issue of purpose. Ultimately, however, the programs we as a nation implement depend on decisions, explicit or implicit, about goals. Eventually, then, we must confront the ambiguities raised in Section One and decide what capacities are necessary for adults in our society and what institutions are responsible for passing them on.

If the problem is one of deficiencies in basic skills--defined as the capacity to do arithmetic, to read simple passages for comprehension, and to carry out simple writing tasks such as filling in applications and forms--then the goals of remedial programs are relatively simple. To be sure, the methods for attaining these goals remain unclear; the efficacy of skills and drills remains to be determined, and the many alternatives, including functional context training, still need to be assessed; but at least the intended outcomes can be readily measured.

The first complexity is to consider whether the issue is less one of basic skills than of work habits. The volume of complaints about the lack of discipline, tardiness, and absenteeism among employees has increased, and many education and training programs have introduced "employability skills" into their curriculum. But when we shift the focus of remediation from basic academic skills to work-related behavior, the outcomes--presumably behavior on the job defined as appropriate by the employer--are affected by much more than prior training, including motivation, wages, working conditions, and the nature of supervision. The real issue is not whether training programs can teach individuals to show up on time, work diligently, and obey their supervisors in jobs with decent earnings, good working conditions, stable employment, and reasonable rules. Instead, the question is whether they can induce such good behavior in the low-wage jobs, with unstable employment and little intrinsic satisfaction, for which short-term programs like JTPA and welfare programs (and some short-term vocational education as well) prepare people. There may in fact be a serious problem with the discipline of the labor force, as with the discipline of students in schools; but if remedial programs are asked to be accountable for behavior which has many other causes outside their control, then it becomes imperative to face the limits of what education and training can do.

Raising the question of work discipline leads to yet another complexity. Work habits are not essentially cognitive skills in the sense that the skills and drills approach assumes reading and math to be; they involve interpersonal behavior. Similarly, the recent reports calling for greater capacities in listening, negotiation, the ability to work cooperatively in groups, and other interpersonal skills are raising questions about skills that again are not cognitive, but involve interaction with others. Here is where the skills and drills approach--with its emphasis on individual and decontextualized learning, sometimes, as in the case of computer-based programs, without any interaction whatsoever with other people--is at the greatest disadvantage because there is absolutely nothing in these methods that develops interpersonal skills. The alternatives to skills and drills, on the other hand, insist that knowledge cannot be divorced from its social setting and that it requires interpersonal interaction to make sense of anything; these approaches, therefore, have substantial advantages in teaching interpersonal capacities. If the basic problem in this country is the interaction of individuals at work, then our fundamental approach to education in general, as well as remedial education in particular, must be abandoned in favor of teaching methods that stress rather than eliminate interpersonal dimensions.

But we can complicate the problem even further. If the challenge is, as A Nation at Risk stated, a need for the New Basics, or if the United States requires a set of "foundation skills" at "world-class levels" to confront the stark choice between "low wages or high skills," then low levels of cognitive skills are insufficient. In this case, the goals of current remedial programs--which typically aim to increase test scores by one or two grade levels, or to prepare their students to pass the GED--are woefully inadequate, and the elementary and secondary education system itself must be completely remade because only a small fraction of its students complete twelve years of schooling with good command of the New Basics or foundation skills. At this point, of course, it becomes necessary to confront the inadequacy of resources in the existing remedial system: The fiscal limitations which have plagued adult education, JTPA, and welfare programs and the time limitations imposed on remedial programs have undermined programs with even limited ambitions, and are clearly inadequate once we escalate the demands we place on our education and training system.

If we acknowledge that much higher levels of cognitive capacities may be necessary, there is an interpersonal and behavioral dimension as well. If, as some commentators and commission reports have claimed, the problem is a lack of judgement on the part of workers, then we are in serious trouble. Judgement--which requires the ability to weigh conflicting purposes and to evaluate the relative importance of economic, moral, political, and interpersonal demands on any particular decision--requires the most subtle combination of cognitive and interpersonal capacities. It is hard to know how to train for judgement, especially in a system which strips any initiative and responsibility from the lower levels of education and training. Furthermore, the failures in judgement are ubiquitous: If the low- and middle-level workers of this country seem to lack judgement, so do our political and business leaders whose exploits of dishonesty, greed, poor business decisions, and simple selfishness have accelerated over the past two decades to the detriment of politics and business alike. If good judgement is what we are after, we have a long way to go.

A final complexity involves the question of who should benefit from reforms. If we take many recent commission reports at face value, the major challenge is to enhance the productivity of the labor force in relatively low-skilled jobs. That is, employers should be the major beneficiaries of enhanced education and training in the form of higher productivity, rather than employees gaining through higher wages or more stable employment, or a greater capacity for political participation, or enhanced capacities as parents or neighbors. This, in turn, leads to the current situation: As we discuss Workplace Basics: The Skills Employers Want, we generate long lists of the skills required in employment and introduce relatively narrow and employment-specific forms of training. In this discussion, what individuals want from adult education and remedial education is nowhere considered. In the first instance this represents a simple inequity, since the interests of employers are represented but those of employees are not. But it also suggests a source of ineffectiveness as well, since it violates a given of adult education--that programs must meet the needs of adults if they are to be effective. Since so many of the skills and capacities required in the labor force--including work-appropriate behavior, cooperation and teamwork, and judgement--involve motivation, it is difficult to promote the interests of employers through narrow training programs while denying the interests of employees. Over the long run, overly specific training or remedial programs that fail to meet the needs of those enrolled will suffer high dropout rates and low enrollment rates--like many current programs--and will serve the interests of no one. Such programs will fail to provide a labor force adequate to the demands of the future, requiring another generation to invest in their own round of reform.

If we take seriously the current storm of interest in the education and training of the labor force, then, we must move beyond the remedial programs concentrating on simple arithmetic and reading that we now have. Instead, we must contemplate much more sophisticated reforms that reshape the K-12 education system as well as remedial programs, that change the nature of teaching throughout the system, and that provide much more intensive forms of education to larger fractions of the population than is now the case. This is, of course, a huge undertaking, and one whose cost has not been considered in any way. Such a proposal seems unrealistic in a period when the federal government continues to run enormous deficits, and when most state governments are facing fiscal constraints of their own, and for this reason alone the question of purpose is one whose resolution will be a long time coming.

But such issues are decided every time a decision about remediation is made. The plans to extend yet another skills and drills adult education program, to experiment within a community college developmental education program, to fund a new program of workplace literacy programs without considering their pedagogy or requiring substantial evaluation, or to establish an experimental program following the whole language approach--all these plans embody implicit decisions about purposes. The current system of remediation is one in which there has been an implicit decision to aim for a low level of skill improvement in simple reading and arithmetic, to prepare individuals for entry-level jobs at close to the minimum wage--jobs which cannot realistically make them self-sufficient. This is a goal that--as far from attainment as it is--is still severely limited compared to the ambitions that have been established by recent commission reports calling for a world-class workplace. As a nation, we need to confront these decisions about purposes explicitly, rather than leaving them to the whims of convention and financing. The simpler issues of coordination, evaluation, and pedagogy cannot be fully resolved until we do so.

[62] Many of our recommendations are consistent with those of Chisman (1989), though his recommendations do not stress the issues of evaluation and pedagogy as strongly as we do.

[63] In a sense, a structure for doing this has been created in places where there has been an effort to centralize all employment and training funds and to allocate them among providers in a rational way. This has been done, for example, in Hartford, and it was the intent behind the Massachusetts effort to establish centralized control over education and training funds in Massachusetts; see Grubb et al. (1990).

[64] See for example, the model in which the community college dominates all vocational education and job training in Grubb and McDonnell (1991).

[65] See, for example, Goodman, Goodman, and Hood (1989); see also the sharp interchange between McKenna, Miller, and Robinson (1990) and Edelsky (1990) on the need for evaluation of whole language approaches and the form such an evaluation might take.

[66] The only partial exception involves the discussion of teacher professionalism. Professionalized teachers would have more autonomy to develop their own methods, and so professionalism is consistent with novel approaches to teaching--but those in favor of greater professionalism (e.g., Holmes Group, 1986) have never clarified how greater autonomy will lead to different and improved teaching.

Grubb, W. N., Kalman, J., Castellano, M., Brown, C., & Bradby, D. (1991). Readin', writin' and 'rithmetic one more time: The role of remediation in vocational education and job training (MDS-309). Berkeley: National Center for Research in Vocational Education, University of California.