Better Predictors of Effective
Lawyering
By Marjorie Shultz,
J.D. and Sheldon Zedeck, Ph.D.
 |
| Shultz |
 |
| Zedeck |
By a show of hands, how many
employers would like a reliable crystal ball when selecting new associates for
their firms, to help determine how likely the candidates are to be effective
lawyers? Wow … that’s a lot of hands.
Recent research conducted at
the University of California, Berkeley, offers methods to appraise a wide range
of individual competencies related to lawyering. The newly designed methods
evaluate a much larger range of abilities than traditional grade and test-score
assessments do; lawyers need classroom-type smarts, but they also need a lot of
other skills not taught in school: problem solving; advocacy; creativity;
self-development; negotiation; practical judgment; stress management;
interviewing and counseling, just to name a few.
In the research, conducted
between 1999 and 2008, more than 2,200 lawyers participated during Phase I in individual
interviews, focus groups and surveys to determine what factors lawyers believed
important for professional effectiveness. Participants also supplied real-life examples
to create scales attuned to actual lawyers’ behaviors so that an individual’s professional
performance could be rated in terms of those factors.
Based on this information, researchers
developed a new test battery that assesses the identified competencies. During
the Phase III validation process, approximately 1,100 graduates of Berkeley Law
and Hastings College of the Law (graduating between 1975 and 2006, practicing
in a variety of fields and settings) took the new test, and granted access to their
academic credentials. After taking the new test battery, those who participated
also provided contact information for two peers and two supervisors who could
evaluate their current work performance on 26 effectiveness factors identified
earlier in the research.
Researchers next compared each
test-taker’s scores on the new test battery with the ratings by self, peer and
supervisors of on-the-job performances. Correlations were significant enough to
demonstrate that a wide array of lawyering skills can be predicted. The new
test battery showed significant correlations (validity) in predicting 24 of the
identified 26 factors whereas traditional academic indicators showed significant
correlation with only eight of the 26, with several of those correlations being
negative. For example, higher academic grades and test scores appeared predictive
of lower competence in networking and business development. The most highly
predictive segments of the new test battery can be completed online in less
than an hour.
The best point in the
pipeline to administer the new tests would be at application to law school. The
research was initially directed toward expanding what admission officers
consider in choosing among applicants. Based on this initial exploratory
research, inclusion of professional effectiveness predictors would help align the de facto gateway to the legal profession with production of lawyers who
are more effective, especially in the early years of practice. If legal
employers use similar tests in making hiring decisions, better legal service
for clients and society should result.
Use of professional
effectiveness tests could also have another significant impact: it could meaningfully
increase racial and ethnic diversity in the profession. As expected, based on
experience in employment where these methods were pioneered, the Shultz-Zedeck
research results were – for all practical purposes – race and ethnicity
neutral. Several decades of experience with similar methods in employment decisions
have shown that selection systems relying primarily on academic grades and
tests create significant adverse impacts on underrepresented minorities; by
contrast, merit-based selection methods that assess competencies actually
required on the job greatly reduce or even eliminate such adverse impacts.
Even in a state as diverse as
California, racial and ethnic diversity in the legal world has only increased
at a glacial pace. That may be true partly because of deeply entrenched
selection traditions that mainly pay attention to academic indicators. Expanding
the types of assessments used in choosing among law school applicants, and in
hiring and promoting lawyers, could make a real difference for diversity.
The research discussed here was
exploratory, aimed to demonstrate a new approach to selection for the legal
profession. Further research and validation are important; nevertheless, the
results of this initial, empirical study are sufficiently strong to suggest
that two important goals of the legal profession – improved effectiveness and
greater diversity – could be advanced by adding these new methods to those by
which we currently choose people for training and practice.
We urge these new methods as “additions”
because academic skills remain relevant in both education and employment. Users
of professional effectiveness testing could combine it with academic indicators
in a variety of ways: allocating varied weights to different types of
information; using grades and test scores to set a floor for cognitive
performance and considering professional prediction tests for decisions among
those who make the cut; or choosing only to emphasize some from among the 26
professional factors according to needs, goals, and so forth.
The legal professions of other
countries are already using work-competency testing, and other professions in
the United States are exploring the potential for such testing in admission to
their professional training programs. For example, US medical schools recently
announced plans to broaden selection criteria among applicants by adding
communication and leadership skills to the scientific knowledge that has historically
been the focus of MCAT testing. The Graduate Management Admission Council is currently
researching which competencies are important to success in MBA programs and
subsequent employment, and whether those can be better predicted by inclusion
of tests other than those encompassed in the GMAT.
The law should not linger at
the back of the 21st century curve in introducing newer and more
comprehensive methods of selection and advancement within its professional
ranks.
To learn more about this
research and its potential applications, please contact Marjorie Shultz
(510)642-1921 mshultz@law.berkeley.edu or Sheldon Zedeck (510)642-7130 zedeck@berkeley.edu).
For a more detailed description of the research, see 36 Law & Social
Inquiry 620 (2011).
Marjorie
M. Shultz is a professor emerita of the UC Berkeley School of Law, and she has
written and consulted in the fields of contracts, health law and bioethics,
race and gender and legal ethics. She is co-author of “White-Washing Race; The
Myth of A Color Blind Society” (U.C. Press, 2003).
Sheldon
Zedeck, Ph.D., is a professor emeritus of the psychology department at UC
Berkeley. He retired in 2010 after more than 42 years at Cal, including almost
four years as Vice Provost for Academic Affairs and Faculty Welfare.
Any opinions and/or viewpoints contained in this article belong solely to the author(s) and are not necessarily the opinion of the California Bar Journal or the State Bar of California or its staff and employees.