Page 2 of 3

Existing methods of selection, both objective and subjective, also exclude people based on their race and gender. For example, although women as a group perform worse than males on the SAT, they equal or outperform men in grade point average during the first year of college, the most common measure of successful performance. Similar patterns have been detected in the results of the ACT and other standardized college selection tests.11

Supplementing class rank with the SAT also decreases black acceptances and black enrollments.12 Studies show that the group of black applicants rejected based on their SAT scores includes both those who would likely have failed and those who would likely have succeeded, and that these groups offset each other. Consequently, the rejection of more blacks as a result of using SAT scores "does not translate into improved admissions outcomes. The SAT does not improve colleges’ ability to admit successful blacks and reject potentially unsuccessful ones."13

Thus it is incontestable that the existing meritocracy disproportionately includes wealthy white men. Is this highly unequal outcome fair? Even if the "meritocracy" screens out women, people of color, and those of lower socioeconomic status, it could be argued that those screens are fair if they serve an important function. But the testocracy fails even on this measure; it does not reliably distinguish successful future performers from unsuccessful ones, even when supplemented by additional subjective criteria. Therefore, racial, gender, and socioeconomic exclusion cannot legitimately be justified in the name of a flawed system of selection.

A New Approach

We have seen how the stock affirmative action narrative normalizes and legitimates selection practices that are neither functional nor fair. Now it is time to use these criticisms as an occasion to move from affirmative action as an add-on to affirmative action as an occasion to rethink the organizing framework for selection generally.

Such rethinking should begin by reconsidering the connection between predetermined qualifications and future performance. The standard approach proceeds as if selection were a fine-tuned matching process that measures the capacity to perform according to some predetermined criteria of performance. This assumes that the capacity to perform–functional merit–exists in people apart from their opportunity to work on the job. It further assumes that institutions know in advance what they are looking for, and that these functions will remain constant across a wide range of work sites and over time.

But neither candidates nor positions remain fixed. Often people who have been given an opportunity to do a job perform well because they learn the job by doing it. Moreover, on-the-job learning has assumed even greater significance in the current economy, in which unstable markets, technological advances, and shorter product cycles have created pressures for businesses to increase the flexibility and problem-solving capacity of workers. Under these circumstances, access to on-the-job training opportunities will contribute to functional merit–the opportunity to perform will precede the capacity.

The concept of selection as a matching process also presumes that institutions have a clear idea of what they value, and of the relationship of particular jobs to their institutional goals. Even in a relatively stable economic and technological environment, institutions rarely attempt to articulate goals, much less develop a basis for measuring successful achievement of those goals. But without a definition of successful performance, it is difficult to develop fair and valid selection criteria and processes.

Defining successful performance has also become more complicated in the current economic and political environment. Traditional measures of success, such as short-term profitability, do not fully define success, and may in fact distort the capacity to evaluate and monitor employee performance. In addition, standards must increasingly change to adapt to technological developments and shifting consumer demand. Students of economic organization and human resources now emphasize the importance of developing complex, interactive, and holistic approaches to measuring both institutional and individual performance.14 Conventional matching approaches to selection do not easily accommodate this move toward more dynamic and interrelated assessments of successful performance.

Current selection approaches also focus on the decontextualized individual, who is assumed to possess merit in the abstract and to demonstrate it through a test or interview. Social science evidence shows that the testing environment can selectively depress the test performance of highly qualified individuals.15 And individual performance does not take into account how an applicant functions as part of a group. Increasingly, work requires the capacity to interact effectively with others, and the demands of the economy are moving in the direction of more interactive, team-oriented production. The capacity to adapt to rapid changes in technology, shifts in consumer preferences, and fluid markets for goods requires greater collaboration at every level.16 Paper-and-pencil tests do not measure or predict an individual’s capacity for creativity and collaboration.

Assessment through opportunity to perform often works better than testing for performance. Various studies have shown that "experts often fail on ‘formal’ measures of their calculating or reasoning capacities but can be shown to exhibit precisely those same skills in the course of their ordinary work."17 Those who assess individuals in situations that more closely resemble actual working conditions make better predictions about those individuals’ ultimate performance. Particularly when those assessments are integrated into day-to-day work over a period of time, they have the potential to produce better information about workers and better workers.

Moreover, many of those who are given an opportunity to perform, even when their basic preparation is weaker, catch up if they are motivated to achieve. Indeed, a recent study of a 25-year policy of open admissions at the City University of New York found that the school was one of the largest sources in the United States of undergraduate students going on to earn doctorates, even though many of its undergraduates come from relatively poor backgrounds and take twice as long to complete their bachelor’s degree.18

Reclaiming Merit and Fairness

Critics of affirmative action defend prevailing selection practices in the name of meritocracy and democracy. We have argued that those practices put democratic opportunity fundamentally at risk. Even when they are modified by a commitment to affirmative action, current modes of selection jeopardize democratic values of inclusiveness (no one is arbitrarily shut out or excluded); transparency (the processes employed are open and are functionally linked to the public character or public mission of the institution); and accountability (the choice of beneficiaries is directly linked to a public good). The failure of existing practice to achieve inclusiveness is perhaps the most telling. Although some people will lose as a result of any sorting and ranking, a democratic system needs to give those losers a sense of hope in the future, not divide us into classes of permanent losers and permanent winners. But that is precisely what happens when we make opportunity dependent on past success.

How, then, can we develop a model of selection that expresses a more inclusive, transparent, and accountable vision of democratic opportunity–an approach to selection that will benefit everyone, and advance racial and gender justice?

An Emerging Model

Because of the importance in a democracy of ensuring opportunities to perform, we can start by shifting the model of selection from prediction to performance. This model builds on the insight that the opportunity to participate helps to create the capacity to perform, and that actual performance offers the best evidence of capacity to perform. So instead of making opportunity depend on a strong prior showing of qualification, we should expand opportunities as a way of building the relevant qualifications.

To follow this model, organizations need to build assessment into their activities, integrate considerations of inclusion and diversity into the process of selection, and develop mechanisms of evaluation that are accountable to those considerations. The result would be a dynamic process of selection, with feedback integrated into productivity. At the level of individual performance assessment, it would mean less reliance on one-shot predictive tests and more on performance-based evaluation.

One fundamental change resulting from our framework would be a shift away from reliance on tests as a means of distinguishing among candidates. Tests would be limited to screening out individuals who could not learn to perform competently with adequate training and mentoring, or be simply discontinued as a part of the selection process. Of course, decreasing reliance on tests to rank candidates would create the need to develop other ways of distinguishing among applicants. There is no single, uniform solution to this problem. One approach would be a lottery system that would distribute opportunity to participate among relatively indistinguishable candidates by chance. Concerns about a lottery’s insensitivity to particular institutional needs or values could be addressed by increasing the selection prospects of applicants with skills, abilities, or backgrounds that are particularly valued by the institution. A weighted lottery may be the fairest and most functional approach for some institutions. Particularly in the education arena, where opportunity lies at the core of the institution’s mission, a lottery may be an important advance. Above that test-determined floor, applicants could be chosen by several alternatives, including portfolio-based assessment or a more structured and participatory decision-making process.19

A more institutionally grounded approach might work in non-educational contexts. In some jobs, for example, decision-makers would assume responsibility for constructing a dynamic and interactive process of selection that is integrated into the day-to-day functioning of the organization. Recent developments in the assessment area, such as portfolio-based and authentic assessment, move in this direction. These might build on the tradition and virtues of apprenticeship, and indeed might "more closely resemble traditional apprenticeship measures than formal testing."20 They would build from and acknowledge the effects of context on performance and the importance of measuring performance in relation to context.

To take the next step in developing an experience-based approach to opportunity and assessment, it would be necessary to consider the needs, interests, and possibilities of the particular institutional setting. The central challenge is to develop systems of accountable decision-making that minimize the expression of bias, and structure judgment around identified, although not static, norms. For each assessment, decision-makers would articulate criteria of successful performance, document activities and tasks relevant to the judgment, assess candidates in relation to those criteria, and offer sufficient information about the candidates’ performance to enable others to exercise independent judgment.

For this model to work, institutions would also need to change the relationship between race, gender, and other categories of exclusion to the overall decision-making process. Institutions would continue to assess the impact of various selection processes on traditionally excluded groups. But institutions would use that information in different ways. Rather than operating as an add-on, after-the-fact response to failures of the overall process, race and gender would serve as both a signal of organizational failure and a catalyst of organizational innovation. We will return to this issue later, but let’s first try to imagine what this more integrated approach would look like.

Consider the case of Bernice, now the general counsel of a major financial institution. Initially, she was hired as local general counsel to a bank, after having previously been partner in a prestigious law firm. (She left the firm after reaching the glass ceiling, unable to bring in enough new clients to progress further.)

Bernice ultimately became general counsel to a major national corporation that previously had no women in high-level management positions. Her promotion resulted from the opportunities presented in an interactive and extended selection process. Her local bank merged with a larger company. In part to create the appearance of including women, she was permitted to compete for the job of general counsel for the new entity. Three lawyers shared the position for nine months. She initially did not view herself as in the running for the final cut.

During this time period, Bernice had a series of contacts with high-level corporate officials, contacts she never would have had without this probationary team approach. As it turned out, Bernice was able to deal unusually well with a series of crises. If standard criteria, such as recommendations and interpersonal contacts, had been used to select a candidate, it is doubtful Bernice would have been picked. But teamwork, decentralized management, and collaborative and flexible working relationships allowed her to develop the contacts and experiences that trained her. The opportunity to interact over a period of time allowed her to demonstrate her strengths to those who made promotion decisions. Bernice did not know she had those strengths until she took the job.21

Now, as general counsel, she is positioned to expand opportunities for women, and corporate culture in general. She can structure the same kind of collaborative decision-making in selection that provided her the opportunity to work her way into the job. She determines who is promoted within the legal department, and who is hired as outside counsel. She is also in a position to influence how women are assessed as managers within the company.

This story illustrates the potential for integrating concerns about diversity into the process of recruitment and selection. It also shows the value of using performance to assess performance. At the core of this integrative move is a functional theory of diversity animated both by principles of justice and fairness (the inclusion of marginalized groups and the minimization of bias) and by strategic concerns (improving productivity). It is crucial to this integration that decision-makers and advocates understand and embrace a conception of diversity that comprises normative and instrumental elements. In public discourse, diversity has become a catchall phrase or cliché used to substitute for a variety of goals, or a numerical concept that is equated with proportional representation.22 Too often, the different strands of diversity remain separate, with those concerned about justice emphasizing racial and gender diversity as a project of remediation, and those concerned about productivity emphasizing differences in background and skills. Without an articulated theory that links diversity to the goals of particular enterprises and to the project of racial justice, public discussion and public policy-making around race and gender issues is more complicated.

Selection and Productivity

One argument for more closely integrating selection and performance is that doing so has the potential to improve institutions’ capacity to select productive workers, pursue innovative performance, and adapt quickly to the demands of a changing economic environment. The conventional top-down approach short-circuits the capacity of selection to serve as a mechanism for feedback about an institution’s performance and its need to adapt to changing conditions. It also keeps institutions from developing more responsive, integrated, and dynamically efficient selection processes. (continued)

Page 2 of 3

[back] [top] [next]