| Oregon State Bar Bulletin — NOVEMBER 2011 |
|
|
Dear desperately under- and unemployed lawyers who graduated from Oregon law schools between 2008 and 2010:
Don’t you wish you’d invested your time and your money in training for a profession in which 87.6 to 91.9 percent of graduates were employed within nine months of graduation?
Of course, if you had checked out your law school on its website or in U.S. News & World Report, you would have been led to believe that you were putting yourself in that position.
Which is why law school graduates and others are questioning where these statistics came from, and learning — among other things — that some law schools actually have paid unemployed graduates to work, rendering them “employed” on key reporting dates.
Data-reporting entities and law schools are pointing fingers at each other, and some graduates of law schools in other states have even sued their alma maters for false representation.
“Jane Smith” (not her real name), a 2010 Lewis & Clark Law School alumna who was paid to work over the key reporting date of Feb. 15, says that she finds these numbers reported for post-law school employment “really hard to believe, even at the lowest number.” (The 87.6 percent figure quoted above is from NALP, the Association for Legal Career Professionals for 2008 graduates; the 91.9 percent is for 2010 graduates.)
“For recent graduates, I would estimate that for people working in the legal field, not in coffee shops, 65 to 70 percent would be high, high,” says Smith, who subsequently found full-time legal employment, then lost it in a round of lay-offs. “And I would estimate that most of those just found jobs in the last few months, and most of those were temporary or term-limited jobs.”
Elizabeth A. “Libby” Davis, associate dean for Career Services and Alumni Relations at Lewis & Clark, acknowledges that the figures most widely reported for post-graduate employment are “raw numbers.”
But these raw numbers, to paraphrase a critic, don’t distinguish between graduates employed as barristers and those employed as baristas. Which is why, says Davis, this issue has become “the big conversation nationally.”
It’s a conversation that also is taking place in other professions whose graduate schools’ statistics also are reported in U.S. News’ annual Best Schools rankings.
“We take part in the U.S. News surveys,” Cynthia Morris, who is responsible for medical-school admissions at Oregon Health & Science University, told the Bulletin. “But probably you know that their surveys are highly controversial. Frankly, their methodology is flawed, absolutely. Many [at professional schools] voice the opinion that we would all like to stop, but until everyone does, we reluctantly submit.”
Phylis Myles, director of Career Services at Willamette University College of Law, says basically the same thing about law schools.
“[The information in] U.S. News is out-of-date,” she says. “The American Bar Association’s information is more up-to-date but people, unfortunately, use U.S. News to make decisions about where to go to law school. So even thought law schools don’t like it, they have to report to U.S. News.”
Who Reports What to Whom?
Let us state, right here, that the point of this article is not to point figures at, or to compare, Oregon’s three law schools. It is to track how the issue of post-law school employment became, as Lewis & Clark’s Libby Davis put it, “the big conversation nationally” (which did not happen, as it might appear, overnight) and where that has left recent law-school graduates.
Law schools, including Oregon’s Lewis & Clark (LC), the University of Oregon School of Law (UO) and Willamette, have collected information on post-graduate employment for years, both for their own and prospective students’ use and to report to other entities.
The best known of the data collection and reporting entities are the ABA, U.S. News and NALP, which goes by the acronym for its former name and bills its 30-plus years of reports as “the only comprehensive study available on the employment experiences of recent law graduates.”
But what most prospective law-school students, graduates and the public didn’t realize, until recently, is that when the ABA/U.S. News/NALP counted the number of graduates who were “employed” as of the key reporting date of Feb. 15 — nine months after a presumed May graduation —those figures were, as OHSU’s Morris put it, “highly controversial.”
“What I tell all of our students,” says Willamette’s Myles, “and even prospective students, is that it (the percentage reported by U.S. News and others) is the top number. When you dig underneath, employed graduates may be doing part-time work, contract work or seeking other work. A lot more are going into solo practice. It’s not as pretty.”
How This Became So Controversial
Had the inability of recent law-school graduates to find legal work not become so overwhelmingly obvious in recent years, this wizard behind the curtain might never have been seen.
And, had prospective students and law-school graduates attempted to see the wizard earlier, the curtain was such a fabric of statistics, often reported in dense, narrative form, that even a suspicious and statistically savvy reader would have had trouble getting at the truth.
But we — being suspicious, although not particularly statistically savvy— will try.
A major problem is that different law schools, and even different data-reporting entities, do not collect and report post-graduate employment data the same way. In fact, the same school or entity may not even do it the same way itself from year to year. This makes comparisons of numbers between one law school and another — unless they are wildly disparate — virtually worthless; comparisons of one data-reporting entity’s numbers with another’s is problematic, and even comparison of the same school’s or entity’s data with its own data from one year to the next is suspect.
“The formulas by various organizations we report to have changed over the last 10 years,” says Rebekah Hanley, the UO’s assistant dean for Career Planning and Professional Development. “For example, what you do with graduates you can’t find or who are pursuing LLMs, etc.: that formula has changed three or four times over the last 10 years. All law schools (based on directions from NALP, the ABA, etc.) used to give employment credit to both groups.”
In addition, as University of Colorado at Boulder Law School Prof. Paul Campos said in an April article in The New Republic, NALP does not report, on an individual-school basis, a percentage that excludes law-school graduates who are employed in non-legal and/or part-time jobs. And U.S. News does not report even this percentage even for all reporting schools combined.
“This means,” wrote Campos, “that the only school-specific information currently available to students is extremely misleading.”
Another problem with post-graduate employment statistics is that they invite prospective law-school students to look to the past to decide their futures.
“Students do ask questions about these (post-graduate employment) figures,” says LC’s Davis. “But it’s hard because you’re looking at something really current, when you will be looking for jobs a couple of years from now.” For example, it’s difficult to believe that less than three and one-half years ago, or just shortly before 2011’s graduates started law school, NALP put out a press release trumpeting that the market for new law graduates was at its highest level – 91.9 percent of those whose employment status was known – in 20 years.
Yet another issue is that the most-widely reported figures, those that appear in U.S. News, are not, as Davis puts it, “really current.”
For example, when U.S. News ran a story captioned “The 2012 U.S. News law school rankings are out!” in March, the story noted, farther down, that the rankings “technically” were based on post-graduate employment data on 2010 graduates’ employment as of Feb. 15, 2011.
According to The New York Times, post-graduate employment is the most-competitive category in U.S. News’ law school rankings, accounting for about one-seventh of the ranking.
Finally, another problem, says Davis, is that “It’s hard for students, when starting law school, to look at these kinds of figures in terms of their employability or marketability.”
“Independent surveys find that most law students would enroll even if they knew that only a tiny number of them would wind up with six-figure salaries,” The New York Times reported in January. “Nearly all of them, it seems, are convinced that they’re going to win the ring toss…”
Classes of 2007 and 2008: From “Vast
Majority” Employed to “Market Shrinks”
With these caveats in mind, let’s go back to NALP’s 2008 report on 2007 graduates, the one that concluded, “The vast majority of Class of 2007 law school graduates — 91.9 percent of those for whom employment status was known — were employed as of Feb. 15, 2008.”
In hindsight, there were hints, even back in 2008, that this percentage wasn’t what it appeared to be.
First, as NALP’s press release acknowledged, the 186 ABA-accredited law schools that had reported figures to NALP had been unable to obtain post-graduate employment information from seven percent of their graduates. This is important because, as The New York Times said in January, “…the results are skewed because graduates with high-paying jobs are more likely to respond than people earning $9 an hour at Radio Shack. Those who don’t respond are basically invisible, aside from reducing the overall response rate of the survey.”
Second, NALP’s press release didn’t say whether the 76.9 percent of the known graduates which it reported as having jobs for which bar passage was required were employed full-time or part-time, in long-term or temporary positions, doing or not doing the kind of presumably legal work that they wanted to do.
And that, unless they plan on assuming a national average of $100,000 in law-school debt with the desire to do part-time contract work, is exactly what prospective students want to know.
Classes of 2008 and 2009: the Slide Begins
One year after this 91.9 percent high-water mark, NALP reported that the “overall employment rate” for the class of 2008 had decreased for the first time since 2003, to 89.9 percent of the graduates whose employment status was known. (This figure was based on reports from 188 ABA-accredited law schools and did not include the employment status of 6.9 percent of that class.)
The next year, for 2009 graduates, NALP’s press release stated that “The overall employment rate of 88.3 percent…represents a 3.6 percentage point drop from the recent historical high of 91.9 percent for the Class of 2007.” (This percent was based on 96 percent of graduates from 192 accredited law schools.)
“The job market for the Class of 2009 was very different than it was for the classes that immediately preceded it,” the release quoted NALP Executive Director James Leipold as saying. “Members of this graduating class were more likely to be working part-time, working in a temporary job, working in a job that does not require a JD, working as a solo practitioner, or working but still looking for another job, than their peers who graduated in the classes before them.”
“…[W]e have,” said Leipold, announcing a major shift, “gone to great lengths to try to measure and describe some of those differences.”
The report also noted that the percentage of graduates employed in academia was the highest ever recorded by NALP.
“…[M]uch of the increase may be accounted for by schools’ efforts to provide post-graduate job opportunities in a tight job market,” it said. “Some of these efforts predate the recession but may have expanded in light of the recession.”
In other words, law schools, out of altruism, guilt and/or a desire to have their graduates count as employed on the all-important date of Feb. 15, were paying some of them to work.
The Controversy Becomes Public
Shortly after NALP’s report on 2009 graduates was issued, Adjunct Prof. Jason M. Dolin of Columbus, Ohio’s Capital University Law School published what appears — from the Bulletin’s web search — to be the first public condemnation of the role of law schools in supplying misleading statistics.
“With the recession lingering into the new year, law school applicants need to know the facts about post-graduation job prospects,” Dolin wrote in the Ohio State Bar Association’s November/December 2010 magazine. “But are universities and the NALP telling the whole story? Unfortunately, some of the data the law schools present are incomplete and misleading. More unfortunately, law-school applicants believe it. What is driving this?”
The answer, said Dolin, is the “tremendous financial pressure” on law schools and their deans to get tuition and fees to make themselves self-sustaining; pay law professors’ relatively high salaries and, in some cases, help support their schools’ other, money-losing academic programs.
“Telling applicants the truth about their real employment prospects might not be good for business,” Dolin said bluntly. “The law schools’ perceived need to ‘pretty up’ their employment data is driven largely by their never-ending need to bring in more and more tuition and, as we will see, to move up the U.S. News rankings ladder. There is no better way to do this than by using NALP’s ‘overall employment rate.’ ”
Dolin then went on to explain, in exhaustive, narrative detail, why NALP’s 88.3 percent overall employment rate for 2009 graduates was “misleading.”
The bottom line, said Dolin, is that by counting graduates “employed in any capacity in any kind of job” as employed, both NALP and U.S. News arrived at grossly inflated figures.
“…[W]hat is the ‘real’ (JD-required, fulltime, non-temporary) employment rate for the national law school Class of 2009?” he asked. “We cannot know with certainty, but based on this data (his calculations from NALP’s/U.S. News unreported data), it is not a stretch to say it is most likely somewhere in the upper 50 to the lower 60 percent range. That is a far cry from NALP’s overall 88.3 percent rate and a further cry from the even-higher employment rates commonly claimed on law-school websites.”
Dolin then moved from dissecting the numbers to law schools themselves.
“Law schools enjoy the cover of NALP’s ‘overall’ employment number, which allows them to claim breathtakingly high — and entirely misleading — employment rates for their graduates,” he wrote. “…In addition, many law schools are notorious manipulators of the data they submit to U.S. News in an attempt to move up in the rankings. …A very small change in a school’s employed-at-nine-months data can produce a drop of 10 or more places in the U.S. News rankings. The practical effect of this system is that law schools have a strong incentive to use Enron-like accounting methods to prop up their employment numbers.”
“[W]hat law students and prospective students want to know,” Dolin concluded, “is very reasonable: how will their $100,000+ legal education enhance their overall employment prospects? This is an empirical question worth answering. And in the output-based world we are now entering, it is all that really matters.”
It Pays to be Counted
Around the same time that Dolin’s article was published in the November/December 2010 issue of Ohio’s bar magazine, an interesting thing happened here in Oregon.
On Nov. 17, 2010, LC graduate Jane Smith and other unemployed graduates received a spritely worded email from Libby Davis captioned: “Graduate Stipends & RA Opportunities.”
“Dear Graduates:”, the email began. “I am pleased to be writing to you with some great news — the Dean has made available a significant fund to be used to assist our December 2009, May 2010, and summer 2010 graduates. The money will be used to fund a number of graduate stipends with government and public-interest organizations AND on-campus faculty research assistant (RA) positions. Each stipend and RA award will be in the amount of $2,500 and (is) to be used to support doing approximately 10 weeks of work. The purpose of the fund is to assist grads both financially, as they seek permanent employment, and experientially.”
According to Davis, LC’s offer was not uncommon.
“More and more,” she says, “law schools are doing what they call bridge stipends. They have become very common because of the economic challenges right now. In 2010 we funded 20 such positions); in 2011, 40 (that were) part-time for eight weeks. The way they were set up was to provide funding to work in a position after the bar exam. They were part-time on purpose, partly to allow us to fund more and to allow them time to look for jobs.”
But what the email, and graduates’ subsequent dealings with LC, didn’t reveal was that this 10-week, LC-funded job would cover the school’s crucial, Feb. 15, 2011 reporting date.
“Basically everybody was covered for the same (10-week) period,” says Jane Smith. “You had to apply by Nov. 28, 2010: they selected within a week or two and allowed us three weeks to find an organization.”
“No one said anything about job reporting,” Smith continues. “I didn’t even know when law schools reported those numbers. The way it was reported to me was the law school felt bad because nobody was getting jobs.”
Which was, says Smith, a reasonable concern.
“At that point, I could count the number of grads I knew with jobs on my two hands,” she says. “I knew hardly anyone with a job: it was abnormal for people to beemployed. I knew several people who were employed at places like at The Gap or at restaurants, but it’s hard to get a job if you have a degree you aren’t using. I applied for a part-time medical records position at a hospital because I had done work like that in college. I was told that if I was going to apply for jobs like that, I shouldn’t list that I have a law degree. I didn’t get an interview.”
Erick Hoffman, director of communications at the UO, says that that law school has offered what he describes as privately funded “post-graduate fellowships” since 2006, although “A decision has not been made on if we will do so in the future.”
Hoffman says that the fellowships, which are for graduates who have letters of commitment from public-interest or public-service employers, pay those graduates to work for three months in what otherwise would be unpaid positions. Hoffman says that while these positions “tend to span” Feb. 15, the timing is dictated by when graduates have passed the bar in Oregon or elsewhere, not by reporting schedules.
And Willamette University’s Myles says that while Willamette does not offer such “bridge” grants, “We do have loan-repayment assistance programs, and we occasionally havehad faculty hire graduates to work for them on books, etc. A couple of years ago an alumnus gave us some money to give graduates contract work with other attorneys. When they have been out of work for so long, and they’re so down, they’re so grateful that someone thought they were good enough to do contract work.”
The Controversy Explodes
Shortly after Jane Smith was awarded her stipend, the
controversy over statistics concerning post-law school employment exploded.
In mid-December 2010, the ABA held a hearing on whether it should refine its survey questions to get more realistic and useful statistics for U.S. News’ rankings.
And in January, The New York Times published an article in which it noted that U.S. News’ rankings are based entirely on unaudited surveys conducted by law schools and asked U.S. News if it unilaterally could demand better data from law schools?
“Yes,” it quoted Robert Morse, U.S. News’ director of data research and the developer of the methodologies and surveys for its annual rankings, as saying.
“But,” Morse qualified, “we’d have to create a whole new definition of ‘employed,’ and it would be awkward if U.S. News imposed that definition by itself. It would be preferable if the ABA took a leadership role in this.”
The following week, Morse used his own U.S. News column, “Morse Code,” to address the issue.
“A recent story in The New York Times…is creating a great deal of buzz in the legal community, at law schools, and among current and prospective law school students,” he wrote. “It’s clear that some law schools are being very aggressive in their reporting practices and are, in effect, gaming the rankings via their accreditation data.”
But, Morse reiterated, “It is very important to remember that U.S. News is using the standards and definitions of employment that have been developed and are used by the NALP and ABA. In other words, U.S. News is reflecting the standard set by the legal industry… It’s not U.S. News’s role to set industry definitions and regulate data quality. If the ABA and NALP adopt new reporting standards and definitions of what it means to be employed, U.S. News would readily follow those new standards and definitions.”
Class of 2010: “Worst Job Market Since Mid-1990s”
On June 1, NALP issued its most-recent annual report, headlined “Class of 2010 Graduates Faced Worst Job Market Since Mid-1990s.”
“…[T]he overall employment rate for new law school graduates is, at 87.6 percent, the lowest it has been since 1996...,” the report said glumly, basing its statistic on 95 percent of graduates from 192 reporting law schools. “In addition, the Class of 2010 employment data reveal a job market with many underlying structural weaknesses, and the employment profile for this class marks the interruption of employment patterns for new law-school graduates that have been undisturbed for decades.”
In the report, NALP Executive Director Leipold explained and defended how NALP calculates its statistics, noting that historically it has based its “overall employment rate” on graduates whose employment status is known and has counted all types of jobs as employment.
While Leipold described this as “a standard practice in calculating employment rates used, for example, by the U.S. Bureau of Labor Statistics,” he acknowledged that there are “alternative” methods.
The issue was reported on again in August,whenFortune magazine said that the ABA would start holding law schools responsible for the completeness and accuracy of their data. However, it said, “Prospective law students will have to wait some time before they can benefit from any of these new requirements, as the first such questionnaire won’t go out until in February of next year, and the questions themselves are still up for debate.”
Even then, U.S. News’ data research director Morse wrote on Sept. 1, don’t expect completely accurate numbers.
“The American Bar Association’s initial efforts to collect and then eventually publish more robust data on post J.D. employment and job placement data are falling far short in some cases,” he wrote, basing his conclusion on the ABA’s 2011 questionnaire for 2010 graduates and a July 27 ABA memo.
While Morse called the new questions “the ABA’s first effort to significantly improve the accuracy, transparency, and usefulness of law placement data,” he accused it of “lagging on what is still needed:” specifically, whether the reported employment is full or part-time, professional or non-professional.
“Without (this information),” Morse notes, “students can’t determine the likelihood that their new J.D. degree will get them a full-time job in the legal field or a part-time position or a job where going to law school didn’t matter. The ABA says it will add these questions for the class of 2011 graduates, but not for the 2010 graduates.”
However, Morse said, “The ABA is going in the right direction in some areas.” These include now asking law schools to report whether employed graduates are working in positions with definite terms of more or less than one year, and — for the first time — whether the employment is, directly or indirectly, law-school funded.
The latter, said Morse, is “…very important to know since it will show how many new J.D. grads are getting jobs in the real world.”
Finally, Morse said that U.S. News is “…studying the possibility of asking the questions that the ABA is not asking in terms of full-time and part-time employment and J.D.-required job or non-J.D. required job.”
“There is some likelihood,” he concludes, “(that) we will use this new jobs data to change the methodology used to calculate employment rates for the 2013 edition of the Best Law Schools rankings, to be published in 2012. Until U.S. News finishes collecting the new data, we are unable to specify how the methodology will change.”
Who’s Working Now?
In an effort to determine how many recent law-school graduates actually are employed, the Bulletinconducted its own little experiment.
We took the list of the 160 law-school graduates who passed the February 2010 Oregon bar exam, figuring that they should have had enough time, if the published post-graduate employment statistics are anywhere close to accurate, to find some kind of work.
We then selected one name for each letter of the alphabet — there were none whose last names began with I, Q, U, V, X or Z — and looked up the remaining 20 names in the current, online bar directory.
Of these 20 names, 16 of them, or 80 percent, were listed, and four were not.
Of the 80 percent that were listed, nine (or 56.25 of the 80 percent) had email addresses that implied they were working for someone other than themselves, either law firms or other professional-sounding employers, like banks or insurance companies.
And, of the 80 percent that were listed as members of the bar, seven (or 43.75 percent of the 80 percent) had email addresses that implied they either were in solo practice, not working in law or not working.
So what is the “overall employment rate” for this group who passed the bar exam? Is it the 56.25 percent, which may include some who passed and are now working for law firms, banks and insurance companies in non-law positions? Does it include some portion of the 43.75 percent whose email addresses may indicate that they are practicing law on their own? And what percentage of this percentage, whatever it is, are working part-time instead of full-time, in temporary or contract positions instead of permanent ones?
And what could have happened to the missing four who passed the bar?
“Since the economic downturn, an awful lot of applicants who [pass are] unemployed,” says Jon Benson, executive director of Oregon’s Board of Bar Examiners, offering one possible explanation. “An increasing number who pass the bar don’t immediate take the oath of office, to avoid paying bar dues.”
Meanwhile, all three Oregon law schools currently have references to U.S. News’ rankings on their websites, and all three still have nine-month post-graduate employment information on their websites and/or otherwise made available to prospective students and graduates. And 2010 LC graduate and stipend recipient Jane Smith continues to search for full-time legal work.
ABOUT THE AUTHOR
Janine Robben has been a member of the Oregon State Bar since 1980 and is a frequent contributor to the Bulletin. She is the legal director of the Oregon Crime Victims Law Center.
© 2011 Janine Robben