Yet, despite its crucial role in these processes, there has been little systematic analysis of the effectiveness and consequences of occupational analysis methods (O'Brien, 1989; Rayner & Hermann, 1988; Wills, 1993a). Moreover, there has been virtually no discussion of how the occupational analysis process and its outcomes affect workers and the roles that they play in the organization.
In this section, we examine the occupational analysis methods used by the skill standards projects. We first describe the process used by the majority (nineteen)[11] of the pilot projects--DACUM (Develop A Curriculum)--and discuss its strengths and weaknesses. We then provide some additional insight into the process as it is used in the pilot projects by taking a look at some of the alternative occupational analyses methodologies. The discussion focuses on whether the method of occupational analysis used is more consistent with the skill components or professional model and whether it promotes the development of workers for high-performance workplaces.
DACUM is facilitated primarily by educators and was developed as a way of bringing business and industry into the development of educational programs. Robert Norton (1993), who developed the technique, states that DACUM is a "significant technique for initiating needed cooperation (between business and education) in tech prep . . ." (p. 1). The process, in effect, functions as an "abbreviated version" of the widely known Functional Job Analysis (FJA) process (Wills, 1993a, p. 3-13).
Expert workers and supervisors are brought together for two-day focus group (brainstorming) sessions or "workshops" to interact, describe their jobs, and rate activities according to their frequency and importance. Workshop participants in the original DACUM structure are "not hampered or constrained by a literature base or any instructor-created document" (Norton, 1993, p. 1), but, rather, are given a blank slate in which to define and describe their occupations. From these focus group sessions, a profile chart is created which details and graphically displays the duties and tasks involved in a particular occupation. Output is submitted to a larger group of workers and/or immediate supervisors for verification. Task-specific curricula are then developed based on the component tasks that the process has determined and verified.
Nineteen of the twenty-two pilot projects used a modified DACUM process (one of the nineteen used a "straight" DACUM process and four used a combination of DACUM and V-TECS[13] methods) for job analysis. Although most of the DACUM methodology remained intact, projects modified the job analysis process in one of three ways. Four of the projects reported that workers and/or supervisors (workshop participants) were asked to validate existing industry standards instead of using the "blank slate" specified by the original DACUM model. Six projects started with output from an extended search[14] of existing industry standards, library databases, curriculum guidelines, instructional material, and current industry/occupational task lists. They then used the DACUM process to validate this output. The remaining eight used a clean slate or free-wheeling job analysis process similar to the original DACUM methodology. But these were "modified" by either including more experts in the process; developing a set of structured interviews to clarify and discuss the outcomes, their phrasing, and the terminology used; adding a mail survey; or visiting sites and observing workers. Overall, project modifications were almost all based on a perceived need for more "structured coordination" than emphasized in the original DACUM process.
Twelve of the nineteen projects that used a modified DACUM methodology organized focus groups as the primary vehicle to determine and validate the tasks and duties, five of the projects used focus groups and site visits, and the remaining two projects used written surveys or questionnaires to solicit responses. All of the projects "validated" their standards. This was done by asking workers and supervisors to comment on the standards (either by written survey or focus-group participation) and suggest any changes or deletions.
The modified DACUM format, while establishing some methodological "ground rules," gave projects the flexibility to customize the process according to particular industry or occupational needs while inciting a sense of comfort that individual approaches taken were not extensively different. This customization, while being potentially problematic in the future when cross-industry standards and occupational clusters begin to emerge, gives industry an opportunity to "own" their job analysis process and, thus, the ensuing standards. Several of the project managers reported that the industry participation facilitated by the DACUM process was crucial to the development of the project.
DACUM proved to be a job analysis approach that was easily understood by both educational and industrial participants. Overall, DACUM has been widely accepted by the educators and industry leaders involved in the skill standards projects. Despite a few minor process-oriented difficulties, DACUM has generated an overall sense of comfort and accuracy surrounding the process and the results that have been produced.
It should be emphasized that although there appear to be some advantages to DACUM, systematic evaluations have not shown it to be superior. Indeed, no clear conclusions have been drawn in regard to the most effective job analysis methodologies (O'Brien, 1989; Rayner & Hermann, 1988; Wills, 1993a). Researchers have voiced difficulty in the evaluation of any job analysis method "due to the difficulty of finding appropriate criteria against which effectiveness can be measured . . . (as well as) the difficulties in defining the occupational area, and in ensuring that each technique is used with a matched representative sample" (Rayner & Hermann, 1988, p. 48).
Indeed, most job analysis methods, by breaking down jobs into their specific component parts, reduce worker roles to a series of unrelated job functions. There is a strong similarity between this approach and the conceptualization of jobs used in a Tayloristic system of job design (Wills, 1993a). Sydney Fine, after developing Functional Job Analysis in the 1960s, argued that most occupational analysis methodologies failed to provide an adequately integrated description of jobs.
The worker needs to know what kind and how much freedom of choice he may exercise as he performs his task and by what standards his performance will be judged. . . . [He] needs to know what in his work is prescribed and what is discretionary. (Fine & Wiley, 1971, p. 19)This statement may have more force today than it did in 1971, given the extent that firms have shifted to high-performance systems where workers have more responsibility and autonomy. To the extent that workers have become "professionalized," job analysis must not only represent the tasks that workers are expected to perform but specify the depth and breadth of their skills by identifying the situations and circumstances that call for them. Work must be placed in a broader organizational context that also relates to the ultimate objective of the work. As the seven steps outlined earlier make clear, DACUM and similar job analysis methods focus attention on the component tasks rather than the broader context and objectives of the work and the worker.
The task focus can also be seen in the V-TECS (Vocational-Technical Consortium of States) process, another job analysis method which was influential in at least four of the skill standards projects and is viewed by many to be similar to DACUM. Indeed, program operators recognize that V-TECS and DACUM are much the same. V-TECS produces task-based output such as duty and task lists; performance objectives for each task; standards as an observable measure of performance; and sequential task performance steps. V-TECS outcomes also include enabling competencies and related academic skills--basic essential skills taxonomy, criterion-referenced test item banks, and performance/ psychomotor items (Wills, 1993a).
Officials who are developing skill standards on the state level have voiced concern as to whether job analysis processes that are commonly being used today (DACUM in most cases) will produce standards that are broad and flexible enough to accommodate changing workplace requirements. At the same time, the standards must be specific enough to be useful to employers and clear enough to be understood by the general public, most of whom have not developed a professional lens for looking at occupations. While DACUM attempts to include the input of many stakeholders in its process, its focus on narrow, traditional occupational classifications and its use of a task-based approach may prevent standards from reflecting the industry or occupation as a whole (Ganzglass & Simon, 1993). If the industry is not reflected broadly, it is unlikely that the occupations within the industry will be given the latitude to move out of their traditional, skill components framework.
A structured, task-oriented approach such as DACUM forces participants to focus on the details of jobs without dealing with the broad, underlying goals of the industry or the occupations within it. One project director referred to this as a "whiskey bottle" methodology in which every minute aspect of the job is passed around the table for discussion, often in a haphazard manner, in hope that the most important responsibilities and duties will eventually emerge. In abiding by a process where outcomes are established in a vacuum (without organizational conditions to frame the nature of the work to be performed), dissention can occur among those with different perceptions of what end result is desired. Furthermore, an individual's perceptions regarding job characteristics, if allowed to develop in isolation from workplace characteristics and dynamics, may not necessarily correlate with actual job attributes (Capelli, 1992).[15] These potential problems were noted by several project directors who discussed the tension between educators and industry representatives (employers, supervisors, and front-line workers) and supervisors and front-line workers in task and skill listings. In determining the most desired occupational skills, there is no context in which to support the opinions of either "side." Educators, it was indicated, see an employee as a lifelong student and seek to establish a base of broad-range skills and knowledge for workers to build upon. Employers, on the other hand, are interested in the specific skills that are in more immediate demand. What employers (and many supervisors) want are skills that can be applied effectively by their workers. Although some employers may want broad generic skills, the needs of employers are often communicated, without a context, as narrowly defined specific skills.
Validation of existing standards or an extended search for past information/sources of standards (which took place in half of the projects) poses an additional danger of embedding the job analysis system even further in traditional conceptualizations of work. Elements of the past and present could become the benchmarks for the future. Hanser (1995) discusses this in terms of official and "emergent" skills. He states that traditional job analysis processes are static and appear to be "more of a snapshot of a job than an organic image of a job" (p. 10), thus focusing more on the current, stated task requirements and not on the dynamic aspects of the job and its contribution to the organization, the worker, or the work performed. While it is easier and perhaps more efficient to edit and alter what is already available instead of starting from scratch, this may not produce the kind of skills or worker profiles that projects indicate they are looking for. Even starting from scratch using a very narrow job analysis framework, as was the case in half of the projects, has a strong conservative bias, especially for those jobs that do not currently assume high-performance characteristics.
DACUM was designed to minimize time and resources in the job analysis phase. Project modifications were carried out to further simplify what can become a difficult, labor intensive, and time consuming process. Nevertheless, one of the most widely cited problems was that the process (particularly the focus groups) took too much time and money. Many directors who used focus groups (both "free wheeling" and more focused) stated that the groups required additional organization and direction to be optimally efficient and productive. Many of the projects found it necessary to modify the focus group structure after their first round. Project directors stated that the original facilitators, most often educators, did not have the technical expertise and knowledge required to answer questions or clarify issues raised by worker/supervisor experts. Project directors then turned to industry representatives to direct the job analysis process.
Managers also reported that they had difficulty finding employers willing to release their workers to participate in focus groups. Others worried that the workers, drawn primarily from among the employees of firms belonging to the relevant employer organization, may not be representative of workers throughout the industry. Indeed, two sets of researchers cite the difficulty of achieving a representative sample of job incumbents for analysis in their review of DACUM (Rayner & Hermann, 1988; Willet & Hermann, 1989). But a search for a more diverse set of workers would have taken more time and resources. Given the prominent role played by employer associations, project directors had much greater access to the employees of member firms.
In general, logistical issues involved in coordinating business and education communities were common. Many of DACUM's coordination problems were addressed by reducing the worker role in the process and, in the interest of time and control, streamlining the job analysis effort by putting more control in the hands of a select group of traditional decisionmakers (i.e., managerial personnel rather than workers).
Much broader approaches to job analysis have also been used for nonprofessional jobs. Unlike the narrowly focused, task-specific data that seem to arise from DACUM and V-TECS processes, the Functional Job Analysis (FJA) methodology (used to develop the Dictionary of Occupational Titles) investigates jobs based on a broad functional scale of how workers relate in seven categories: (1) Data Functions--complexity in the use of information; (2) People Functions--level of interpersonal skills demanded; (3) Functions That Involve Using Objects (Things)--physical requirements, typically with machines; (4) Worker Instructions--level of responsibility; (5) Reasoning Development--from common sense to abstract undertakings; (6) Mathematical Development--math skills; and (7) Writing Functions (Capelli, 1992; Fine, 1988).
The Comprehensive Occupational Data Analysis Program (CODAP), developed and used extensively by the military, requires special computer programs to analyze statistically its extensive task inventory and vast background gathered on job incumbents, including their career aspirations, educational level, tools and equipment used in previous work experiences, work attitudes, and prior training.
The Position Analysis Questionnaire uses 187 worker-oriented job elements to characterize the human behaviors that are involved in jobs, not simply the tasks that are being performed. Among its six broad categories are Information--where and how one gets information needed for the job; Mental Processes--reasoning, decision-making, and planning activities that employees use; Work Output--physical activities, tools, and so forth; Relationship with Other People--measures of complexity; Job Context--physical and social parameters of work; Other Job Characteristics--irregular work schedules and repetitive activities (Capelli, 1992; McCormick, 1979).
The Critical Incidence Technique identifies hundreds or thousands of critical incidents that illustrate effective or ineffective (successful and unsuccessful) job-related behaviors as a vehicle for determining the aims or purposes of the job. Occupational Analysis Inventory (OAI) consists of 622 work elements grouped into five categories: information received, mental activities, work behavior, work goals, and work context. The Job Information Matrix System (JIMS) gathers and records job information into categories such as the responsibilities of the worker and the working conditions of the job. The Threshold Traits Analysis System focuses on workers rather than the work itself. Worker traits are categorized as either physical, mental, learned knowledge and skills, or social (Capelli, 1992). Hay Associates developed a measure for skill similar to the DOT measure, The Hay Associates Profile System, which focuses on three areas: know-how (capabilities, knowledge, and techniques needed to do a job); problem solving (thinking demands of the job, scaling tasks as to whether they would be considered repetitive and routine, or requiring adaptive abilities for abstract concepts and ideas); and accountability (amount of autonomy in decision-making, amount of guidance, and amount of impact that individual decisions will have on the organization) (Capelli, 1992). Similarly, the U.S. Department of Labor's O*NET (Occupational Information Network) has developed cross-job descriptors that detail job-specific information using six categories: worker requirements, worker characteristics, experience requirements, occupational requirements, occupation-specific, and occupation characteristics. In this framework, occupational requirements include generalized work activities, organizational contexts, and work conditions; occupational characteristics include labor market information, occupational outlook, and wages.
Cost and practicality are perhaps the most serious drawbacks to these ambitious techniques. These approaches are long and tedious and often require specially trained personnel. The DACUM process is seen as more user-friendly and appeals to the collaborative nature of the current skill standards development movement by including (at least in theory) both educators and industry representatives. On the other hand, these more focused data do not allow for the same breath and depth of analysis that broader data accommodate. The characteristics of the data may have a more direct tie to the current purpose or aim of the analysis but the immediate gains of more focused data may be difficult to sustain in the future as the focus on the analysis changes and key pieces of data (or aspects of the job) that were considered unnecessary (and therefore uncollected) become important.[16]
Although the more broad-based methods collect data that could be used to move away from the skill components framework, standards actually established using such data look very much like those based on DACUM or V-TECS--a specialized occupational profile that describes workers by identifying a list of their skills. Most of the broader occupational analysis methodologies discussed above include the contextual situation and other relevant aspects of the worker in the data they collect; nevertheless, they fail to incorporate these broader, external, social aspects and definitions of the job into the analysis. This can be seen by examining the Dictionary of Occupational Titles, which provides detailed listings of job characteristics for over twelve thousand jobs. This is clearly not consistent with the view of work based on broader conceptualizations of occupational clusters that we have argued is more consistent with the professional model.
Furthermore, although the content produced by these broader approaches may have been at least potentially more consistent with the professional model than the output of DACUM, the broader perspectives failed to provide any significant role for workers (and, therefore, failed to live up to the professional model in terms of the governance structure). Indeed, while DACUM calls for the participation of workers in focus groups, Functional Job Analysis and similar approaches produce occupational profiles by using outside experts to observe and report on workers--workers are not involved in the job analysis process, nor in the validation phase, much less in the actual development of standards.
In this tension between complex content and practical imperatives, at least on a formal level, the practical appears to have won. The job analysis processes that dominate the skill standards projects are much more consistent with the skill components than the professional model, both in terms of the content of the standards (the conceptualization of skill) and the governance structure. The DACUM and V-TECS approaches tend to result in a task-focused list of skills and to marginalize workers in the standards process. The basic DACUM model does not establish a strong leadership role for workers and the "modifications" give them only an advisory role in validating standards developed by project staff or "experts."
In the end, the output of any job analysis technique is only part of the standards development process. As we have seen, it is possible to collect extensive information through job analysis but ignore when standards are created. When workers or professionals who are embedded into a community or practice associated with the occupations being analyzed are integrally involved with the standards development process, the job analysis technique may appear to be more simple and superficial than the standards the job analysis is presumably being used to create. These practitioners bring their own understanding to the process that may not be contained in the formal approach to analysis. But we do suggest that it will be difficult to develop standards consistent with the professional model by using task-oriented job analysis techniques that are set without extensive participation of incumbent workers.
This is not to say that project managers have not struggled with the limitations of the job analysis methodologies. Earlier in the paper we described three broad approaches to defining standards--compartmentalized, contextualized, and consolidated. Certainly those who used the consolidated perspective have gone beyond the confines of the narrow occupational analysis methodologies such as DACUM. But these achievements come despite, rather than as a result of, the occupational analysis techniques that they have used.
[12] Educational programs that coordinate high school and community college curricula and programs.
[13] Another job analysis technique discussed in this paper.
[14] Extended Search, although often considered a stand-alone job analysis technique, originated as an aspect of the Job-Task Inventory Method or the CODAP (Comprehensive Occupational Data Analysis Program) developed by the U.S. Air Force.
[15] Capelli (1992) cites the findings of Myles and Eno that indicate substantial differences in workers' self-reports of skill requirements in their jobs and those provided by expert raters.
[16] We are not endorsing the use of one specific occupational analysis method. Indeed, various authors have listed countless difficulties and pitfalls in using many of the traditional job analysis methods that currently exist (e.g., see Hanser, 1995, and Rayner & Hermann, 1988). Nor do we argue that the DACUM process cannot be used effectively, especially if it is used in conjunction with other approaches. We present some suggestions for approaches to job analysis in the conclusions to this report.