Law School Rankings and Federal Policies Regarding Disclosure About Affordability

29 november 2023

By Derek T. Muller*

Abstract. The affordability and value of higher education are matters of enduring public conversation. Increased scrutiny has led to increased disclosure. And for legal education, the affordability of legal education has been publicly debated. Affordability data has even been included in the rankings of law schools, which have recently faced pushback from law schools themselves. This essay describes the conversation and debate around law school rankings and affordability data, and it offers some ways for federal policy to help improve affordability disclosures for legal education.

I. Recent Changes in Law School Rankings

Law school rankings are powerful. They influence prospective students about which school they should attend, how much they are willing to spend to attend a school, and whether to transfer to a school. They influence faculty, who weigh competing job offers or placement of articles in law schools’ journals. And the U.S. News & World Report (USNWR) rankings, one of higher education’s earliest and most enduring rankers, hold significant sway.

Rankings include objective figures, like the admissions statistics of an incoming class and employment outcomes. Some criteria are subjective—such as academic or professional reputation—and are based on impressions, instinct, generic reputation, or longstanding prestige.[1] And decisions to include or exclude criteria, or to give criteria more or less weight, are subjective. Decisions like which admissions statistics are important, which employment outcomes are better or worse, and how to weigh these figures against one another are judgment calls. Reasonable minds may disagree about what criteria to use.

Law schools have long pursued higher rankings in an effort to secure the most highly credentialed students and attract the attention of the most elite employers.[2] Some admissions decisions only make sense in light of chasing metrics measured by USNWR.[3]

In November 2022, the Wall Street Journal broke the news that Yale Law School and Harvard Law School (in statements by Dean Heather Gerken and Dean John Manning) would no longer “participate” in the law school rankings published by USNWR.[4] To refuse to participate or to “boycott” meant that these schools would not complete the statistical survey that USNWR submitted to the schools. A slew of other law schools followed.[5]

USNWR rankings data relied upon a combination of publicly available data, private data that schools voluntarily shared with USNWR, and proprietary data USNWR collected independently. Publicly available data included information from the American Bar Association (ABA), like incoming class admissions statistics and employment outcomes. Private data included law schools’ financial expenditures per student and debt loads of graduates. Proprietary data included USNWR’s surveys of other law faculty, lawyers, and judges, which yielded “quality assessment” scores of schools ranked on a scale of 1 (marginal) to 5 (outstanding).[6] With a substantial number of schools no longer sharing private data, USNWR announced it would change the data it relied upon, and it would also substantially change the formula for its rankings.[7] Importantly, it would no longer use private data, which was obtainable only through voluntary disclosure from schools. It would rely much more heavily on publicly available information.

II. Public Disclosures and Legal Education

Publicly disclosed information can be more reliable than privately held information, and public disclosure offers important benefits. If the representations are made to an accrediting body, for example, there is a greater incentive for law schools to disclose the information truthfully. There are essentially no material consequences for disclosing misleading information to USNWR, and it might be the case that some schools would seek to get away with sharing misleading information until they are exposed.[8] Lying to an accrediting agency, however, may result in fines and public censure, even threatening the school’s accreditation.[9] Public disclosure not only allows prospective students to consider and readily compare information when making a decision, but also allows schools to compare themselves and ensure they are disclosing truthful information.

The ABA’s Section of Legal Education and Admissions to the Bar accredits most law schools in the United States, and USNWR only ranks schools accredited by the Section. The Section wields significant influence in gathering information to be made available to the public—what to collect and disclose, what not to share, and what to refuse to request in the first place. USNWR now relies principally on that public data in its rankings.

This publicly disclosed information can assist prospective students as they evaluate and compare law schools. It can also be incorporated into future rankings by USNWR or other enterprising entities. Indeed, most of what USNWR uses in its present rankings comes from ABA-provided data.[10]

That said, other valuable information may be available—or could relatively easily be made available—to the public. This information could be disclosed voluntarily, but it is impossible to coordinate that across nearly 200 law schools. USNWR has also stopped using privately collected information after schools began to refuse to share that data with USNWR. That makes public disclosure all the more critical.

Public disclosure is particularly important now that less information about affordability and cost is available to prospective students. In the past, USNWR would collect information about the percentage of students who incurred debt and the average debt load among graduates. Law schools voluntarily provided the information, and USNWR would publish it.

Federal policies and federal agencies can increase the disclosure of relevant information for prospective law students—and that information might also be usefully incorporated into future rankings. There are at least three ways federal policymakers could improve disclosure. First, federal agencies could better organize existing disclosures they make available to the public. Second, federal agencies could disclose information they currently have but do not readily disclose to the public. Third, federal policy could compel additional disclosure.

A look at these three ways of increasing disclosure gives federal policymakers things to consider in improving access to public information relating to higher education, particularly legal education. All of them, along different dimensions, relate to cost and affordability—one of the most challenging areas to evaluate along any single metric, and one of the areas lost when USNWR switched to only publicly available information.

III. Department of Education College Scorecard Disclosures

First, the Department of Education should improve its aggregation and disclosure of data in its College Scorecard. A better format for disclosing information can make the information more accessible to the public.

The College Scorecard was an effort launched in 2013 to offer transparent consumer information to prospective students about prices and values of higher education.[11] In the last decade, it has had limited influence. Nevertheless, the College Scorecard has some value. It aggregates the recent debt and income for graduates at each institution. It breaks down the data for each degree offered at each institution and for each concentration. In other words, it offers publicly available institutional data that could be evaluated by the public and potentially incorporated in a rankings system.

But the College Scorecard data is not user-friendly. For instance, Department of Education data codes various degrees and concentrations. But there is no singular code for a “J.D.” At some schools, this degree is labeled a “doctoral degree.” For other schools, it is a “first professional degree.” Most degrees appear to rely on the label “law” as the concentration for this degree, but “legal studies” or related fields may be used. Some schools have incomplete data in a given year’s dataset, and the Department of Education offers little explanation about why.

Despite these weaknesses in the College Scorecard data, I have tried to assemble the data to compare median debt loads and median incomes of graduates of these institutions.[12] There are, of course, confounding variables and limitations to this analysis. Many borrowers will be eligible for Public Service Loan Forgiveness programs, either at the federal level or at their own law schools. Those programs can substantially reduce the apparent burden of debt over the years. Suppose schools have disproportionately higher percentages of students entering those programs. In that case, their debt levels will appear worse than they actually are, and their salaries will appear on the lower end of the income scale. This effect is another limitation in thinking about a single-figure metric. And medians are likely skewed in other ways: for example, the highest-earning graduates likely received the largest scholarships and, accordingly, graduated with the lowest debt. If possible, disclosure of 25th and 75th percentiles would provide a more complete portrait.

This information is distinct from the data collected by law schools for the ABA. The ABA data only tracks each year’s cohort of graduates—their debt at graduation and their employment outcomes within a year after graduation. The College Scorecard data aggregates multiple years of graduates and includes salary data four years removed from graduates’ graduation date.

The Department of Education already collects and discloses this data. It simply does so in a less-than-helpful format, and some of its data decisions are opaque. The Department of Education should improve the way it compiles and shares this data. However, there are admittedly limitations to this data. What about schools that offer robust loan forgiveness? Debt loads at graduation tend to overstate debt burden for some graduates who opt to participate in public interest work and experience loan forgiveness. To address this, another category of disclosure is useful.

IV. Department of Education Public Service Loan Forgiveness Disclosures

Second, the Department of Education should consider disclosing more Public Service Loan Forgiveness information. Like the first suggestion, this disclosure turns on information already at the disposal of federal agencies. But unlike the first suggestion, it would be a new category of disclosure that should be revealed to the public.

Debt and cost figures can be misleading. Students pursuing public service can see their debt loads reduced significantly. Dean Gerken of Yale made the case this way:

In addition, the rankings exclude a crucial form of support for public interest careers—loan forgiveness programs—when calculating student debt loads. Loan forgiveness programs matter enormously to students interested in service, as they partially or entirely forgive the debts of students taking low-paying public interest jobs. But the rankings exclude them when calculating debt even though they can entirely erase a student’s loans. In short, when law schools devote resources to encouraging students to pursue public interest careers, U.S. News mischaracterizes them as low-employment schools with high debt loads. That backward approach discourages law schools throughout the country from supporting students who dream of a service career.[13]

In other words, both federal government and school-specific programs can dramatically reduce student debt loads. And rather than schools funding students at the admissions stage with large scholarships, schools may want to fund them at the back end by subsidizing those who cannot repay their loans while allowing those entering lucrative private practice to repay.

That’s an attractive system. But there is a potential causation problem here. What if a student accrues significant debt and feels “trapped”? The student might take a high-paying job in a career she does not wish to pursue in an effort to repay those loans.[14] Alternatively, a student may pursue a public interest-oriented career to reduce debt loans but may not find fulfillment in the public service position. The definition of “public interest” does a lot of work. Whether other income-driven repayment programs[15] can supplement this system matters, but such programs may do more to conceal weaker employment outcomes from law schools than the genuine choices of graduates. In short, there’s tremendous complexity in what we should draw from these programs.

Nevertheless, the Department of Education already has this important information at its disposal. Its debt forgiveness programs have been plagued by complexity and failures, and it has made significant changes in recent years to ensure that students with debt can benefit from the programs.[16] As it reforms these programs, it should consider ways of disclosing how the programs benefit students, including those with particular degrees or from particular schools. While this information may be limited, prospective students should understand some of the potential opportunities to reduce their debt loads in the future.

The Department of Education does not have information about school-based loan forgiveness programs at its disposal. It may be worth considering whether it should compel law schools to disclose the size and scope of those programs, perhaps through modifications to the accreditation standards.

V. Accrediting Body Disclosures

Third, the Department of Education should require accrediting bodies to include more information about outputs, specifically financial cost or affordability. This requirement would pressure accrediting bodies to collect and disclose more helpful information to the public. Unlike the first two suggestions, which exert pressure to improve disclosures of existing information, this suggestion adds new information to the public domain.

The required Department of Education standards for accrediting bodies are mercifully minimal. The relevant provisions of the Federal Register and federal law include fairly generic topics.[17] An accrediting agency, for instance, must “set forth clear expectations” in matters like “curricula,” “faculty,” and “student support services.”[18] One of the more detailed, but still flexible, standards is about student success: “Success with respect to student achievement in relation to the institution’s mission, which may include different standards for different institutions or programs, as established by the institution, including, as appropriate, consideration of State licensing examinations, course completion, and job placement rates.”[19] These standards are generic enough to give accrediting bodies like the ABA flexibility to develop criteria to evaluate these measures as they see fit.

It may be fair to interpret this section to include measures about the financial cost or affordability of the legal education program. Perhaps the Department of Education would need to update the Federal Register to include such matters expressly. Or maybe Congress must update federal law to provide for that expressly. This essay takes no position on which path of administrative law is appropriate. Instead, it simply suggests that rather broad criteria like “financial cost” or “affordability” are appropriate measures.

These are broad and flexible standards that the Department of Education could adopt, that the ABA could administer, and that law schools could comply with. Express provisions of cost or affordability would allow institutions to explain the steps they are taking to ensure that school is affordable. Law schools already share some financial information: tuition; the percentage of students receiving scholarships; the 25th, 50th, and 75th percentiles of scholarship awards; and conditional scholarship data (scholarships that are reduced or eliminated if students fail to meet certain academic benchmarks).[20] Schools also disclose extensive details about employment outcomes.

But more specific details about financial cost or affordability metrics could be useful. While schools disclose employment outcomes, salary data would be useful—the twenty-fifth, fiftieth, and seventy-fifth percentile of salaries across employment outcomes. Some scholarship data is disclosed, but other disclosures could separate need-based aid from merit-based aid. Dean Gerken put it this way in relation to the rankings: “A far better measure is how much financial aid a law school provides to its students, rewarding schools that admit students from low-income backgrounds and support them along the way.”[21] Including such measures in the accreditation process would be valuable. The debt loads of graduates would also be a useful figure—again, looking at the twenty-fifth, fiftieth, and seventy-fifth percentiles to show the performance of graduates of the class as a whole.

Of course, any one numerical figure standing alone can be deceptive. Salary figures can be deceptive because they do not necessarily account for graduates’ happiness or preferred employment outcomes, and lower salaries can be more satisfactory with lower debt loads or in regions with lower costs of living. Need-based aid does not necessarily relate to graduation or employment success, and it’s possible that socioeconomic disparities can lead to disparate outcomes for students. Debt loads are not as crucial if salaries are high and tend to rise quickly—and lower debt loads may simply reveal that a school has a wealthier student body. In short, any one of these metrics has its limitations. But more disclosure can undoubtedly reveal important, if limited, insight into institutions of higher education generally, and legal education in particular. And salary information can be counterbalanced with debt and loan forgiveness information. New obligations for accrediting agencies to incorporate when evaluating institutions of higher education would result in the disclosure of more useful information to the public.

VI. Promises and Limitations of Disclosures

Law school rankings continue to draw significant attention from law schools, law students, and prospective students. At their best, they can offer ways of comparing schools against one another on similar benchmarks for students who may not have a straightforward way to compare schools—especially those students without family or employment history in the practice of law. At their worst, however, they reduce schools to a few factors in a ranking set that may be volatile,[22] manipulable,[23] and given undue weight by law school applicants.

It is not clear that disclosure would necessarily lead to improved rankings. To start, USNWR might simply ignore such new data. It would be left to prospective students to decide whether the new data is valuable. Alternative rankings systems, like AccessLex’s “XploreJD,” seek to use more data in more customizable ways for prospective students.[24] It is possible that more robust disclosure of data could offer alternative systems more opportunities to share data in ways that more prospective students find valuable.

And rankings incorporate criteria in ways that schools often contest. Adding data that has its own limitations further complicates the value of these new disclosures. If debt metrics have limitations and are given a particular weight in a new ranking, two new complicating factors have been introduced into the overall rankings.

Admittedly, a major concern about this proposal might be that the game is not worth the candle. The data issues, from collection to public release, might be so significant that they are not worth pursuing. More information could lead to more confusion about which metrics are essential and which are not and could lead to more deception if more figures are taken out of context. Each round of disclosures adds complexity that demands even more disclosure.

But these concerns do not justify disclosing less information. Disclosures may be flawed, but not so flawed as to justify withholding the information. The problems identified earlier with existing disclosures indicate that some remedy is desirable and that some way of improving disclosures should be pursued. And whether or how that information might be used in the rankings remains a topic of robust methodological debate, regardless of the type or format of the data.

Indeed, changes are already happening in both the Department of Education and in USNWR. The Department of Education recently revealed that new, more detailed financial and employment outcome disclosures will be made available to prospective higher education students.[25] In September 2023, USNWR added publicly available debt and salary data for college graduates to its “national universities” ranking in its “Best Colleges Rankings.”[26] Reforms like these may continue and may affect legal education.

Admittedly, any one measure fails to capture the complete portrait of law school outcomes. Cost and affordability turn on many factors. Nevertheless, rather than disclosing little about cost, disclosing several aspects of affordability would benefit students. Prospective students could consider similarly situated schools with these figures and compare them across employment outcomes. Students interested in public interest careers could better understand the debt relief programs available to them. Schools with robust need-based financial aid programs could gain prominence. And maybe these disclosures could be put into a ranking formula in the future—in ways that try to account for the complexity of financial outcomes. More meaningful statistics about cost beyond tuition and outcomes beyond categories of jobs or bar passage rates can assist student decision-making and should be encouraged.