| [Author's note: originally published in the Wilson Library Bulletin, I believe in
September of 1989] ASSESSING
THE ASSESSMENT CENTER
by James LaRue I once knew a librarian who was a master of the job interview. I never knew him to make a hiring mistake. When I asked him how he did it, he said he had a built-in b.s. detector. Some people set it off; he didn't hire them. Picking the right person for the job is crucial to a library's success. Picking the wrong person can be expensive - in lost time and training, even in a lawsuit. It can also be very damaging to staff morale, to a library's credibility within the community or profession, to its finances, or all three. But there's more to the selection process than deciding who to hire. The applicant has a few questions too. Will she be happy in the job? What kind of people run the place? What are they really looking for? For most of us, whether we're on the hiring or the seeking side, the job interview is a hit-or-miss proposition. Sometimes it goes well, sometimes badly. Usually, we don't quite know why. Not all of us are born with b.s.-detectors, nor, alas, are they routinely distributed with MLS diplomas. Library managers need a more reliable personnel selection tool - and that tool is something called the "assessment center." I've been involved in three assessment centers: twice as a candidate, once as an assessor. Those librarians who haven't run across the assessment center yet, soon will. Use of the assessment center is a trend. But the assessment center has its problems too, particularly from the point of view of the applicant. THE TRADITIONAL INTERVIEW Almost all of the hiring decisions in today's business are made after a personal interview. Usually people up for promotion go through an interview as well, most frequently by the potential supervisor. For most librarians, the interview process is very traditional. We meet with one or two other people, ask some questions, get quizzed in turn, and somehow a decision is made. As an approach to personnel recruitment, this method isn't so hot. Some interviewers may use a standard battery of questions for each candidate. Others may not. But even when some standards have been set, ratings are notoriously subjective. It comes down to the judgment and/or biases of the interviewers. They have b.s.-detectors or they don't. Speaking of biases, the traditional interview tends to winnow out minority applicants. The traditional interview is commonly used for three reasons: it doesn't cost much; it seems easy to do; and people are used to it. But as mentioned above, hiring the wrong person can be very expensive. And selecting superlative staff is not particularly easy. Just because people get used to something, doesn't mean it works. If you're interested in making sound hiring decisions - or you want to negotiate your way through a new complex of hiring tests - you need to know about the assessment center. WHAT IS AN ASSESSMENT CENTER? The Task Force on Assessment Center Guidelines has adopted the following definition of the assessment center (and brace yourself for some jargon): "An Assessment Center consists of a standardized evaluation of behavior based on multiple inputs. Multiple-trained observers and techniques are used. Judgments about behavior are made, in major part, from specifically developed assessment simulations. "These judgments are pooled by the assessors in a meeting among the asessors or by a statistical integration process. In an integration discussion, comprehensive accounts of behavior and often ratings of it are pooled. The discussions result in evaluations of the performance of the assessees on the dimensions or other variables which the assessment center is designed to measure. Statistical combination methods should be validated in accord with professionally accepted standards." To be a true assessment center, seven essential elements must be present. Briefly, they are as follows:
A panel interview or a series of sequential interviews does not constitute an assessment center, nor does reliance upon any one technique, even if it's a simulation. If there's only one assessor, it's not an assessment center. If there are many assessors, but they don't pool their data - if they only turn in their own, unintegrated written reports, for instance - it still doesn't qualify. And finally, putting a sign over a room that says "Assessment Center" isn't enough to bring it about either. HISTORY OF THE ASSESSMENT CENTER The quest for a more reliable method of identifying people with high potential is not new. What we now call the assessment center has a long history. It was evolved first for the military, refined for the business community, and eventually applied to the public sector. Psychologists were involved in the selection and training of the German armed forces in 1929. By the late 1930s, a multiple- assessment procedure for officer selection was firmly in place. While ultimately abandoned, and severely limited in many respects, the German programs were much ahead of their time. By World War II, the British War Office Selection Boards picked up on the Germans' earlier efforts, and modified them to great success. The first use of the assessment center in the United States was also during World War II. The Office of Strategic Services (a war time agency set up by the President and Congress to recruit and train spies) both refined and validated assessment center techniques. The first industrial application of the multiple assessment procedure in the United States was conducted by AT&T. Called the Management Progress Study, its purpose was to identify the individual and organizational characteristics that would likely result in a successful manager. The study evaluated 422 newly hired men over an eight year period. This was the most fully realized assessment center to that point, and the follow-up data was most impressive. The assessment process identified eighty- five percent of the individuals who achieved the middle management level. In the jargon of the field, this demonstrated the high "predictive value" of the assessment center. The AT&T study appears to have been the turning point for the assessment center. Bolstered by other industrial and psychological studies, the assessment center achieved more widespread acceptance in industry, and by the mid-1970s, had begun to spread into the public sector. California, one of the nation's bellwether states, has actively promoted the assessment center. Grant money provided by the Intergovernmental Personnel Act established four assessment centers in California cities for the selection of personnel in police and highway patrol departments. The model was adapted and furthered by the California State Personnel Board and was later extended into fire departments as well as such offices as County Personnel Director, City Engineer, Public Works Director, Parks and Recreation Director and Affirmative Action Officer. Probably over 2,000 private and public organizations have used assessment centers. However, the private sector still uses them more than the public sector. Some fear the Californication of the Union. But I predict that the assessment center will one day be as ubiquitous as the dress- for-success seminar. At any rate, public librarians are encountering the assessment center more and more frequently. PLANNING THE ASSESSMENT CENTER As mentioned earlier, some interviews claim to be assessment centers but aren't. The picture I'll draw of the "typical" assessment center is based on the standards listed above. I am indebted also to the counsel of Cynthia Bentson and Vicki Kaman of Ft. Collins, Colorado, both of whom hold Ph.D.s in industrial/organizational psychology and have conducted many assessment centers. The assessment center is an expensive proposition (see "Advantages and Disadvantages" below). As a result, although the assessment center was originally designed to identify managerial potential from a pool of current employees, nowadays its primary purpose is the hiring of top- and mid-level managers. The traditional - and unreliable - hiring techniques continue to be used in order to fill most non-supervisory vacancies. Planning the assessment center is the responsibility of one of two groups: either the city has its own personnel staff or facilitators, or outside consultants are brought in. The first step in the assessment center process is to identify job dimensions. The facilitators or consultants spend many hours with the boss or colleagues trying to get an accurate picture of what the job will entail. A full day job "audit" would not be unusual; half a day might be more common. One purpose of this pre-interview is to come up with a job description. This description will list those qualifications of the job that are utterly necessary. It might also note those qualifications that are desirable and pertinent, but not essential. For example, it may be that computer expertise is a particular concern at present. This step is key to everything that happens next. All of the techniques to follow depend upon one thing: a sure grasp of the job's requirements and the contexts in which they occur. The facilitators must then generalize and abstract from the job description to come up with three or four job dimensions. (Some assessment centers may use as many as fourteen or fifteen, however.) Dimensions are behavioral categories that should define and encompass what the job calls for. Usually, any managerial job will boil down to these two dimensions: administrative (or leadership) and interpersonal skills. Some have added the third dimension of judgment. But each of these might be split into others. The administrative dimension might be divided into time management and team-building, for example. The next major step in planning the assessment center (after creating the exercises, about which more below) is to train the assessors. The training usually takes place over one entire work day. The facilitators' responsibility is to provide an overview of the entire assessment process. Then, they should fully explain what the dimensions of the job are, and define them by specific ranges of behavior. Some candidates will exhibit behavior that's "on target" - they can do the job. Or the candidates might be above or below target. The thrust is again observable behavior. Assessors are severely adjured to record such things as "Candidate said, `We're a family,'" rather than, "boy, is she a good leader!" Or suppose the assessors are observing a role-playing situation with an employee about to be transferred to another position. An above- target candidate might say "The library needs your special training." A candidate who says, "We've all got to do this," might be on target. And the bozo who rants, "If you don't do this, I'll be so mad," is probably below target. Non-verbal behavior is fair game too. Candidates who sit tall and maintain an open posture will fare better than slumped arm-crossers. Finally, the facilitators review the packets the candidates will be working through. The reasoning behind each exercise and in some cases, the inclusion of specific pieces of paper items, is carefully justified. ASSESSMENT CENTER EXERCISES The next major step in planning the assessment center is to create exercises that will stimulate candidates to demonstrate their command of the necessary skills. The facilitators must first choose which techniques to use. Then, they must create the necessary documentation for the candidates and assessors. Assessment centers use a lot of paper. These exercises could involve almost anything. But there are recurrent elements. Perhaps the most common assessment center technique is the leaderless discussion group. I'll give an example of how it might be used for a library job. Six candidates file into a room, are seated at a common table, and given written or spoken instructions. The candidates are all to talk for some specified time - 20 minutes, say - about how a library might increase circulation of periodicals. A second topic might also be tossed in: outlining the steps for a community survey, for instance. Little other direction is supplied. The candidates are just supposed to talk for 20 minutes and try to come up with some good ideas. As they speak, they are silently observed by five or six assessors. What are the assessors looking for? Measurable behavior. Do the candidates maintain good eye contact? Is their body language open, receptive? Or do they sit huddled in their chairs, lashing out at anyone who disagrees with them? If candidates haven't got leadership and communication skills, they probably won't do well in the leaderless discussion. That's precisely the point. Candidates are not being asked to tell someone what terrific leaders they are; rather, they're given an opportunity to demonstrate it as forcibly as possible. The oral resume, also called a self-presentation, is a quickie. Candidates are given a brief period - 5 to 10 minutes - to give a summary of the highlights of their careers. Sometimes candidates are asked to do little charts of their professional lives, or use some other "graphic presentation." How much preparation time is there? It varies. Candidates might get several days notice. Or it might be as little as half an hour. The point here is to test for creativity or presentation skills generally. (Instead of the oral resume, incidentally, or in addition to it, candidates might be asked to give a formal presentation about some library-related topic. In that case, they will probably have been given at least a day or so to prepare for it.) For any presentation, the relevant behaviors might be poise, eye contact with audience, use of humor, or imagination as demonstrated by phrases or objects used. An example of this last might be: "In 1987, my career could best be represented by this Mr. Potato Head. I just happen to have brought a Mr. Potato Head. Note the little cowboy hat." Another common assessment center exercise is the in-basket. And it's wicked. In this one, the candidates are asked to imagine that it's a Sunday afternoon, and nobody else is in the office. There's a pile of papers on the desk and they have just an hour or so to go through all of it and leave appropriate memos or messages to deal with whatever needs attention. If the facilitators have done their job, the in-basket can provide a terrific overview of what kinds of issues the successful candidate will need to deal with. The in-basket is jumbled: some things are there just for the hell of it. Some things are urgent. Sometimes the solution is at the top of the pile; the problem is at the bottom. Sometimes there are conflicting messages. For example, Joe is working on some performance evaluations, and writes that Beth, the circulation clerk, has no interest in her job. But the packet includes Beth's brilliantly reasoned - and unsolicited - analysis of a circulation problem. A linchpin of the assessment center is the simulation or role- play situation. These are carefully orchestrated scenes that last about 15 or 20 minutes. Instead of asking candidates, for example, how well they deal with people yelling at them, somebody hollers at them to see what they'll do. Or perhaps the job will require a lot of work with volunteers; candidates might have to counsel a couple of volunteers with intense personality conflicts. The trend, incidentally, seems to be toward using trained actors as role-players. Various other techniques may also be included in the assessment center. For example, candidates might be asked to participate in management games, or panel interviews, or even to fill out "paper-and-pencil" intelligence tests. Intelligence tests, however, are not usually weighted very heavily in assessment centers. (And that's a curious thing. Only verbal intelligence scores come close to anything like the success rate of assessment center ratings.) THE REST OF THE STORY After all the exercises have been defined and the necessary paperwork produced to support them, the facilitators schedule the day. They plan room assignments, mix up the exercises so the same person doesn't always go first, and so on. Once the assessment center has begun, the facilitator plays the vital role of keeping the session on schedule. This is important. Candidates are wandering bemusedly from room to room, assessors are furiously recording their observations, and somebody has to make sure the right exercise happens at the right time. The final step of the assessment center is to assemble the assessors - usually the day after the exercises - to pool all their observations. There are several ways to go about this. Let's say that on target behavior for leadership is given a point value of "2." Above target might be "3"; below target, "1". Each candidate is discussed for each dimension, for each exercise. Finally, all the assessors must agree on an overall rating. This is not an averaged number; ratings are reached through consensus. If there's significant disagreement about a candidate, the assessors have to marshal their observations and keep arguing until one or the other backs down or some compromise is reached. Then all of these ratings are pooled overall - again, through consensus - to give each candidate a numeric rating. I should note that even the clearly unacceptable people are carefully considered. In part, this clarifies the process for the assessors. It also provides ample documentary evidence for any independent review - should, for example, the Equal Employment Opportunity Commission show an interest in a particular hiring decision. The final outcome is up to the organization's appointing authority. This authority might - or might not - choose to take the recommendation of the assessors. I suspect, however, that appointing authorities usually do. After all, they've paid for the assessment center, committed the necessary staff to it, and if they've been involved in the process at all, understand why the recommendation was made. ADVANTAGES AND DISADVANTAGES OF THE ASSESSMENT CENTER The main advantage of the assessment center is simple: it works. Taken together, the techniques used in an assessment center provide more information about the candidate's probable success in an organization than anything else that's out there. On the management side, that's good; that's what an interview is for. On the employee side, the assessment center has been shown to provide greater opportunity for minorities than more traditional interview methods. If the exercises are closely patterned on the real job, there can be other benefits. The assessors may gain a new appreciation for what the position requires. The candidate can also get a good preview of the work. So if the candidates don't like what they see, they can quietly drop out of the competition. There are disadvantages. The main drawback for the employer is cost. Typically, the assessment center will cost at least $500 to $1000 per candidate. There are staff costs. The assessment center will take at least three days for each assessor: one day for training, at least one day for assessing, and one more for making a decision. On the other hand, the cost of this time may be balanced by the reduction in the risk of hiring the wrong person. There is a more subtle but significant disadvantage. The assessment center was designed to be a staff development and promotion tool. The people being assessed - whether in the military or in a business - already belonged to organizations whose unique goals they understood. When the assessment center is used as a hiring tool, the key element of organizational orientation is absent. This results in a loss of reciprocity in the interview process. The etymological meaning of "interview" is "seeing each other." The assessment center should and usually does result in very accurate insights into candidates' abilities and potentials. The candidate acts and reacts. The assessors observe and record as impartially and impassively as possible. In its design, the assessment center aims to approximate a one-way mirror. That's not "seeing each other." This loss of reciprocity has several negative consequences. For one thing, it sometimes scares off the best candidates. For example, suppose the leading candidate cracks a joke or two. The response of the assessors? - Not even the hint of a smile, no more than a scribbled notation and attentive glance. Who would want to work for such humorless automatons? For another thing, the lack of direct contact between assessor and candidate means that they have no idea how well they'll be able to communicate with each other. They don't know chemistry. Unless the assessment center is based on close-to-fact job situations, candidates could walk away from a grueling day playing stressful and ambiguous games and not have any better idea of what the job entailed, or what their potential boss was looking for, than when they arrived. Interviews should be reciprocal. It's not enough for the assessors to know they want the candidate to work for them. The candidate has to have enough information to decide if she wants to work for and/or with them. The assessment center doesn't provide that kind of information. To that extent, it is not only unfair to job applicants, it's also a waste of their time. THE HUMAN PERSPECTIVE Given its great success in finding the best people for a job, assessment centers can and should be used for the filling of vacancies in library management positions. However, the assessment center process as currently practiced is not sufficient to hook and land qualified applicants. If library managers hire consultants to conduct a true assessment center, they should insist that the process be spelled out, in writing, to the candidates invited to participate - before they show up. It might not be a bad idea to include a citation for this article. As part of the initial introductions, the facilitators should state that the assessors have been instructed to carefully observe and record what they see. The assessors are not robots. They do have senses of humor. This seemingly minor step just might convince some of the better candidates not to write off the process at the outset. Finally, I strongly recommend that all assessment centers conclude with a low-stress, unmonitored, casual opportunity for the candidates to quiz their potential boss. End the interview with a barbecue, or a picnic. This last stage should also be spelled out in the invitation. Tell the candidates to work up a list of their own questions. Then promise them at least half an hour of individual time with their potential boss or bosses. And make sure the candidates get that opportunity. After grilling someone unmercifully for a day or so, it's not fair to ask that person to come to work for a stranger. In the assessment center, as in the press of computer technology, the library comes face to face with its modern dilemma. On the one hand, we have a tradition steeped in the humanities. On the other, we are a gathering place for the potent ideas of science. These ideas transform us. Yet we must preserve the essence of our mission. Whether we deal with the community of our users, or the community within our organizations, let our watchword be service. We are not microscopes; the people who seek to work with us are not bacterial specimens. Let us remember to be polite. BIBLIOGRAPHY Feuer, Dale, and Lee, Chris. "The Kaizen Connection: How Companies Pick Tomorrow's Workers." Training, 26 May 1988, pp. 23-35. Frank, Fredric D.; Bracken, David W.; and Struth, Michael R. "Beyond Assessment Centers." Training and Development Journal, March 1988, p. 65. Gaugler, Barbara B.; Rosenthal, Douglas B.; Thornton III, George C.; and Bentson, Cynthia. "Meta-Analysis of Assessment Center Validity." Journal of Applied Psychology 72 (1987):493-511. Joiner, Dennis A. "Assessment Centers in the Public Sector: A Practical Approach." Public Personnel Management Journal 13 (1984):435-450. Kaman, Vicki S., and Bentson, Cynthia. "Roleplay Simulations for Employee Selection: Design and Implementation." Public Personnel Management 17 (1988):1-8. Klimoski, Richard, and Brickner, Mary. "Why Do Assessment Centers Work? The Puzzle of Assessment Center Validity." Personnel Psychology, 40 (1987):243-259. O'Leary, Lawrence R. Interviewing for the Decisionmaker. Chicago: Nelson-Hall, 1976. Task Force on Assessment Center Guidelines. "Guidelines and Ethical Considerations for Assessment Center Operations." Photocopy. Pittsburgh: 17th International Congress on the Assessment Center Method, 1989. Thornton III, George C., and Byham, William C. "Assessment Centers and Managerial Performance." New York: Academic Press, 1982. Yeager, Samuel J. "Use of Assessment Centers by Metropolitan Fire Departments in North America." Public Personnel Management, 15 (1986):51-64. |