6.1.1 Technology-Assisted Review (TAR)

Used by lawyers on both sides of the “v.,” TAR refers to software designed to streamline and simplify the classification and review of documents, primarily through the use of machine-learning techniques.

Though TAR tools vary in their construction and algorithmic particulars, most operate with some human supervision. Virtually all require lawyers to hand-code, or “label,” a subset of a corpus of documents for relevance or privilege (the “seed set”). Then, those documents are used to train a machine-learning system to categorize additional documents. This process is iterative and may repeat over multiple rounds of labeling and training, until lawyers are satisfied that all documents have been correctly categorized.Footnote ²⁰

Even the most basic forms of TAR represent a big leap from its predecessors. Prior to TAR’s advent, document discovery required lawyers and their non-lawyer staffs to hunch over bankers’ boxes or filing cabinets, and then, in time, to manually flip through scanned documents on computer screens, reviewing thousands or even millions of documents one-by-one.Footnote ²¹ Not surprisingly, the cost of this hands-on review was exorbitant; in 2000, it was estimated that discovery accounted for as much as one-third to one-half of total costs where discovery was actively conducted, and perhaps significantly more in large-scale litigations.Footnote ²²

In the early aughts, both keyword searches and outsourcing came to the fore to address some of the above. But neither proved wholly satisfactory. Keyword searching enabled parties to cut costs by restricting manual review to only those documents containing specific keywords, but search yields were worryingly incomplete.Footnote ²³ Outsourcing – the move to send discovery to less-expensive contract lawyers in out-of-the-way US cities or abroad – was similarly fraught. Supervision was difficult; parties fretted about conflicts, confidentiality, and rules of multijurisdictional practice; and quality was wanting.Footnote ²⁴

As against those halfway innovations, TAR’s advantages are profound. Estimates of TAR’s efficacy vary and are hotly contested, but the general view is that implemented well – and this is a key qualifier – TAR systems are as good as manual, eyes-on review in terms of recall (i.e., the proportion of relevant documents in the total pool of documents that are accurately identified as relevant) but systematically better in precision (i.e., the proportion of documents flagged that are in fact relevant). The far bigger difference is efficiency: Compared to its conventional counterpart, TAR achieves all of this at a fraction of the cost.Footnote ²⁵

Yet, TAR is not without controversy. Much of it stems from the fact that TAR, like any machine learning system, is a socio-technical “assemblage,” not a turnkey engine.Footnote ²⁶ Attorneys must label and re-label documents as the system works its way toward a reliable model. An important implication is that, much like Colossus (described below), TAR systems are manipulable by humans in their construction and tuning.Footnote ²⁷ As Diego Zambrano and co-authors detail elsewhere in this volume, this manipulation can run the gamut from outright abuse (e.g., fudging the labels lawyers apply to document labelsFootnote ²⁸ or rigging the selection, adjustment, or validation of modelsFootnote ²⁹) to a more benign but still respondent-friendly calibration of the system to favor precision (the proportion of responsive documents among those in a production) over recall (the proportion of responsive documents identified).Footnote ³⁰ As a result, and as discussed in more detail below, if litigation’s “haves” need not show their work to the other side, they can shade discovery to their advantage and use their better technology and technologists (if the other side can afford them at all) to make sure it sticks.Footnote ³¹

6.1.2 Colossus

For the nation’s casualty and property insurers, AI has not so much spawned new litigation tools as supercharged those already in use. The best example is Colossus, a proprietary computer software program marketed by Computer Science Corporation (CSC) that “relies on 10,000 integrated rules” to assist insurance companies – the ultimate repeat players – in the evaluation and resolution of bodily injury claims.Footnote ³² Initially developed in Australia and first used by Allstate in the 1990s, Colossus has grown in popularity, such that it has been utilized by the majority of large property and casualty insurers in the United States, including behemoths Aetna, Allstate, Travelers, Farmers, and USAA.Footnote ³³

Colossus has radically changed the process of auto accident claims adjustment. By extension, it has profoundly altered how the tens of thousands of third-party bodily injury claims generated annually by American drivers, passengers, and pedestrians are processed and paid by US insurers.

Before Colossus, an experienced auto accident adjuster employed by Allstate or USAA would have assessed a personal injury claim using rough benchmarks, in a process that was more art than science. Namely, the adjuster would add up a victim’s “special damages” (chiefly, the victim’s medical bills) and multiply those by a fixed sum – often, two or three – to generate a default figure, called a “going rate” or “rule of thumb.”Footnote ³⁴ Then, the adjuster would leaven that default figure with the adjuster’s knowledge and past practice, perhaps informed by a review of recent trial verdict reports, and possibly aided by “roundtabling” among the insurer’s veteran casualty claims professionals.Footnote ³⁵

With Colossus, however, the same adjuster can now calculate a claim’s worth at a keystroke, after plugging in answers to a series of fill-in-the-blank-style questions. Or, as Colossus itself explains: “Through a series of interactive questions, Colossus guides your adjusters through an objective evaluation of medical treatment options, degree of pain and suffering, and the impact of the injury on the claimant’s lifestyle.”Footnote ³⁶ To be sure, the data an adjuster must input in order to prime Colossus to generate a damage assessment is voluminous and varied. When inputting a claim, the adjuster accounts for obvious factors such as the date and location of the accident, alongside the claimant’s home address, gender, age, verified lost wages, documented medical expenses, nature of injury, diagnosis, and prognosis. Treatment – including MRI or X-ray images, prescriptions, injections, hospital admissions, surgeries, follow-up visits, and physical therapy – is also granularly assessed.Footnote ³⁷ Then, against these loss variables, the adjuster must account for various liability metrics. Fault (in all its common law complexity) is reduced to “clear” or “unclear,” while the existence or nonexistence of “aggravating factors” (such as driver inebriation) is also considered, and, in a nod to the tort doctrine of anticipatory avoidable consequences, the adjuster must also input whether the claimant was buckled up.Footnote ³⁸ Even the individual identity of the handling attorney, treating physician and/or chiropractor, and (if applicable) presiding judge is keyed in.Footnote ³⁹

Once data entry is complete, Colossus assesses the claim in light of the enormous pool of data in its master database to generate a “severity point total.”Footnote ⁴⁰ Then, aided by particularized, proprietary information that is specific to each insurer (based on each individual insurer’s “settlement philosophies and claims practice”Footnote ⁴¹), Colossus converts the point total into a recommended settlement range.Footnote ⁴² Insurance adjusters use this settlement range in their negotiations with unrepresented claimants or their counsel. Indeed, at some insurers, adjusters are not permitted to offer a sum outside the range, at least without a supervisor’s approval.Footnote ⁴³ At others, adjusters are evaluated based on their ability to close files within Colossus-specified parameters.Footnote ⁴⁴ In so doing, according to one insider: “Colossus takes the guess work out of an historically subjective segment of the claims process, providing adjusters with a powerful tool for improving claims valuation, consistency, increasing productivity and containing costs.”Footnote ⁴⁵

Beyond these, allegations about further operational details abound. The most common is that, when customizing the software (the proprietary process that converts a “severity point total” into a settlement range), certain insurers “tune” Colossus to “consistently spit out lowball offers.”Footnote ⁴⁶ Some insurers reportedly accomplish this feat by excluding from the database certain figures that, by rights, should be included (e.g., large settlements or verdicts).Footnote ⁴⁷ Others get there, it is said, simply by turning dials downward to generate across-the-board haircuts of 10–20 percent.Footnote ⁴⁸

As such, it appears that, in the hands of at least some insurers, Colossus has not only rationalized the resolution of personal injury claims and injected newfound objectivity, predictability, and horizontal equity into the claims resolution process. It has also systematically cut claims – to the benefit of repeat-play insurers and the detriment of their claimant-side counterparts.

6.1.3 The Walmart Suite

A third innovation combines elements of both TAR and Colossus. One exemplar under this umbrella, which we dub “the Walmart Suite,” given its development by Walmart in partnership with the law firm Ogletree Deakins and in concert with the tech company LegalMation, seeks to rationalize recurrent areas of litigation (think, employment disputes and slip-and-falls). It reportedly operates along two dimensions.Footnote ⁴⁹ First, it reportedly generates pleadings and papers – including answers, discovery requests, and discovery objections – thus cutting litigation costs.Footnote ⁵⁰ To that extent, the Suite might be thought akin to TAR in its ability to perform low-level legal cognitions and generate straightforward work product that previously required (human) lawyers. Second, and more provocatively, the Suite can evaluate key case characteristics, including the identity of plaintiffs’ counsel, and then offer a prediction about a case’s outcome and the likely expense Walmart will incur if the case is litigated, rather than settled.Footnote ⁵¹ The Suite thus seems to be a beefed-up Colossus, with a focus on slip-and-falls and employment disputes rather than auto accidents.

The advantages of such tools are seemingly substantial. LegalMation reports that a top law firm has used its tools to handle 5,000 employment disputes – and, in so doing, the firm realized a six- to eight-fold savings in preparing pleadings and discovery requests.Footnote ⁵² But these economies are only the beginning. Outcome prediction engines, commonly referred to as the “holy grail” of legal tech,Footnote ⁵³ allow large entities facing recurring types of litigation to quickly capitulate (via settlement) where plaintiffs have the benefit of strong claims and talented counsel – and then battle to final judgment where plaintiffs are saddled with weak claims or less-competent counsel. In so doing, the Walmarts of the world can save today by notching litigation victories while conserving litigation resources. But they can simultaneously position themselves over the long haul, by skewing case outcomes, driving down damages, and pushing precedent at the appellate level. We return to these advantages below.

6.2.1 TAR Wars: Proportionality and Discovery Abuse

For TAR, our appraisal is mixed – though the dynamics at play are not simple and our predictions less than ironclad. That said, we predict that the next decade will feature increasingly heated “TAR wars” waged on two fronts: proportionality and discovery gaming and abuse. If, on each front, there is sufficient judicial oversight (an admittedly big if), TAR might usher in a new era, where discovery emerges more efficient and transparent than before. But there is also the possibility that, like Colossus and the Walmart Suite, TAR will tilt the playing field toward repeat-play litigants. Here, we address these two fronts – and also these two divergent possible outcomes.

Proportionality: Will TAR’s efficiencies justify more expansive discovery? Or will these efficiencies yield a defendant-side surplus? Discovery has long been the 800-pound gorilla in the civil justice system, accounting for as much as one-third to one-half of all litigation costs in cases where discovery is actively employed.Footnote ⁵⁴ High discovery costs, and the controversy surrounding those costs, have powered the creation of numerous rules and doctrines that constrain discovery’s scope.Footnote ⁵⁵ One such rule – and the one we address here – is the “proportionality” requirement, that is, a requirement that a judge greenlight a discovery request only if the request is “proportional” to a case’s particular needs.Footnote ⁵⁶

Applied to TAR, proportionality is tricky because TAR can yield gains in both efficiency and accuracy. For a requesting party (typically, the plaintiff), more efficient review justifies more expansive review, including document requests that, for instance, extend to a longer time horizon or to a wider net of document custodians. For a producing party (typically the defendant), however, accuracy gains mean that the requesting party will already get more relevant documents and fewer irrelevant ones, even holding constant the number of custodians or the scope of the search.Footnote ⁵⁷ In short, TAR generates a surplus in both efficiency and accuracy, and the question becomes how best to allocate that surplus.Footnote ⁵⁸

Given these dynamics, judges might employ the proportionality principle in one of two ways. Judges could recognize that the unit cost of discovery – the cost of each produced document – has declined and compensate by authorizing the requesting party’s more expansive discovery plan. If so, the cost of each produced document will drop, transparency into the underlying incident will (at least arguably) improve, and the overall cost of discovery will remain (roughly) constant. Judges, however, might take a different tack. Notwithstanding TAR’s efficiency advantages, judges might deny requesting parties’ motions to permit more expansive discovery, thus holding proportionality’s benchmarks firm. If so, TAR will cough up the same documents as before, but at a discount.

If trial judges permit producing parties to capture TAR’s cost-savings without compensating by sanctioning more sweeping discovery plans, the effect on civil litigation, from the availability of counsel to settlement patterns, could be profound.Footnote ⁵⁹ Lower total discovery costs, of course, might be a net social welfare gain. After all, a core premise of proportionality rules is that litigation costs, particularly discovery costs, are disproportionate to the social value of the dispute resolution achieved, and scarce social resources might be better spent on other projects. But shifts in discovery costs can also have distributive consequences. It is a core premise of litigation economics that “all things being equal, the party facing higher costs will settle on terms more favorable to the party facing lower costs.”Footnote ⁶⁰ If TAR causes discovery costs to bend downward, TAR’s surplus – and, with it, any settlement surplus – will systematically flow toward the net document producers (again, typically, repeat-play defendants).Footnote ⁶¹ Such an outcome would yield a tectonic shift in the settlement landscape – hard to see in any particular case, but potentially quite large in aggregate. It will be as if Colossus’ dials have been turned down.

The potential for abuse: TAR appears to be more susceptible to abuse than its analog counterpart. How will judges respond? The second TAR battleground will be discovery abuse and gaming. The fight will center on a core question: Can discovery rules generate enough trust among litigants to support TAR’s continued propagation, while, at the same time, mitigating concerns about gaming and the distributive concerns raised by such conduct?

Discovery abuse, of course, is not new. Nor is TAR uniquely vulnerable to discovery abuse.Footnote ⁶² Indeed, one of the easiest ways to manipulate a TAR system – the deliberate failure to flag (or “label”) a plainly responsive document – is no different from garden-variety discovery manipulation, in which lawyers simply omit obviously responsive and damaging documents or aggressively withhold borderline documents on relevance or privilege grounds. But, as Zambrano and co-authors note in Chapter 5, there is nevertheless good reason to believe that TAR might be especially prone to abuse – and that is a very serious problem in a system already steeped in mistrust.Footnote ⁶³

TAR’s particular vulnerability to abuse flows from four facts. First, TAR operates at scale. In constructing a seed set, a single labeling decision could, in theory, prevent an entire species of document from coming to light.Footnote ⁶⁴

Second, and relatedly, TAR can be implemented by small teams – much different than the sprawling associate armies who previously performed eyes-on document reviews in complex cases. This means that, in a TAR world, deliberate discovery abuse requires coordination among a smaller set of actors. If discovery abusers can be likened to a cartel, keeping a small team in line is far easier than ensuring that a sprawling network of co-conspirators stays silent. Moreover, TAR leans on, not just lawyers, but technologists – and, unlike the former, the latter might be less likely to take discovery obligations seriously, as they are not regulated by rules of professional conduct, need not participate in continuing legal education, and arguably have a less socialized sense of duty to the public or the court.

Third, TAR methods may themselves be moving toward more opaque and harder-to-monitor approaches. In its original guise – TAR 1.0 – lawyers manually labeled a “seed set” to train the machine-learning model. With access to that “seed set,” a litigation adversary could, in theory, reconstruct the other side’s work, identifying calls that were borderline or seemed apt to exclude key categories of documents. TAR 2.0, by contrast, starts with a small set of documents and uses machine learning to turn up lists of other candidates, which are then labeled and fed back into the system. TAR 2.0 thus renders seed set construction highly iterative – and, in so doing, makes it harder for an adversary or adjudicator to review or reconstruct. TAR 2.0, to invoke a concept in a growing “algorithmic accountability” literature, may, as a consequence, be less contestable by an adversary who suspects abuse.Footnote ⁶⁵

Fourth and finally, while TAR is theoretically available on both sides of the “v.,” technical capacity is almost certainly unevenly distributed, since defense firms tend to be larger than plaintiffs’ firms – and are more richly capitalized. With these resource advantages, if defendants are tempted to engage in tech-driven litigation abuse, they (and their stable of technologists) might be able to do so with near impunity.

The tough question becomes: How should judges react, to safeguard the integrity of discovery processes? Here, judges have no shortage of tools, but all have drawbacks. Judges can, for example, compel the disclosure of a seed set, although such disclosures are controversial, since the full seed set necessarily includes both documents that lawyers labeled as relevant as well as those irrelevant to the claim.Footnote ⁶⁶ Meanwhile, disclosure of TAR inputs arguably violates the work product doctrine, established in the Supreme Court’s 1947 decision in Hickman v. Taylor and since baked into Rule 26(b)(3), which protects against disclosure of “documents and tangible things that are prepared in anticipation of litigation.”Footnote ⁶⁷ And, a call for wholesale disclosure – ever more “discovery about discovery” – seems poised to erode litigant autonomy and can itself be a bare-knuckled litigation tactic, not a good-faith truth-seeking device.

Worse, if analog discovery procedures are left to party discretion absent evidence of specific deficiencies, but a party’s use of TAR automatically kicks off protracted ex ante negotiations over protocols or onerous back-end “report card” requirements based on various quality-control metrics, there is the ever-present risk that this double standard will cause parties to throw up their hands. To the extent TAR’s benefits are overshadowed by expensive process-oriented disputes, investment in TAR will eventually stall out, depriving the system of its efficiencies.Footnote ⁶⁸ Yet, the opposite approach is just as, if not more, worrisome. If judges, afraid of the above, do not act to police discovery abuse – and this abuse festers – they risk eroding the integrity of civil discovery and, by extension, litigants’, lawyers’, and the public’s faith in civil litigation.

Time will tell if judges can steer between these possibilities. But if they can, then out of these two gloomy visions comes a glimmer of light. If judges can help mint and then judiciously apply evenhanded protocols in TAR cases, then perhaps the system could end up better off than the analog system that TAR will steadily eclipse. Civil discovery could be one of those areas where, despite AI’s famous “black box” opacity, digitization yields a net increase in transparency and accountability.Footnote ⁶⁹

6.2.2 Colossus and the Walmart Suite: The Litigation of Losers

When it comes to the slant of the civil justice system, an assessment of the likely effect of Colossus and the Walmart Suite is more dour.

Colossus: Reduction via brute force. The impact of Colossus on the civil justice system seems fairly clear and not particularly contingent. Colossus’ advent has certain undeniable benefits, injecting newfound predictability, consistency, objectivity, and horizontal equity into the claims resolution process. It has also, probably, reduced the monies paid for fraudulent or “built” claims,Footnote ⁷⁰ as well as the odds that claim values will be influenced by improper factors (racial bias, for example).Footnote ⁷¹ Finally, it has, possibly, driven down the driving public’s insurance premiums – though there’s little reliable evidence on the point.

But, alongside these weighty advantages, it does seem that Colossus has also reduced claim payments quite significantly, using something like brute force. When Allstate rolled out a new Colossus-aided claims program for Allstate with the help of McKinsey & Co., the consulting firm’s stated goal was to “establish[ ] a new fair market value” for such injuries.Footnote ⁷² It appears that that aim was achieved. A later McKinsey review of Allstate found: “The Colossus sites have been extremely successful in reducing severities, with reductions in the range of 20 percent for Colossus-evaluated claims.”Footnote ⁷³ Nor was this dynamic confined, necessarily, to Allstate. Robert Dietz, a fifteen-year veteran of Farmer’s Insurance has explained, for example: “My vast experience in evaluating claims was replaced by values generated by a computer. More often than not, these values were not representative of what I had experienced as fair and reasonable.”Footnote ⁷⁴

The result is that, aided by Colossus, insurance companies are offering less to claimants for comparable injuries, on a take-it-or-leave-it basis. And, though one-shot personal injury (PI) lawyers could call insurance companies’ bluff and band together to reject these Colossus-generated offers en masse, in the past two decades, they haven’t.

Their failure to do so should not be surprising. Given persistent collective action problems and yawning information asymmetries (described in further detail below), one would not expect disaggregated PI lawyers, practicing alone or in small firms, to mount a coordinated and muscular response, especially since doing so would mean taking a significant number of claims to trial, which poses many well-known and formidable obstacles. For instance, some portion of PI lawyers operate in law firms (called “settlement mills”) and do not, in fact, have the capacity to take claims to trial.Footnote ⁷⁵ Second, many auto accident claimants need money quickly and do not have the wherewithal to wait out attendant trial delays. And third, all PI lawyers are attuned to the stubborn economics of auto accident litigation: As of 2005, the median jury trial award in an auto case was a paltry $17,000, which would yield only about $5,500 in contingency fees, a sum that is simply too meager to justify frequent trials against well-financed foes.Footnote ⁷⁶ This last point was not lost on McKinsey, which, in a presentation to Allstate, encouraged: “Win by exploiting the economics of the practice of law.”Footnote ⁷⁷

The Walmart Suite and the litigation of losers. The Walmart Suite illustrates another dynamic, which we dub the “litigation of losers.” In the classic article, Why the Haves Come Out Ahead, Marc Galanter presciently observed that repeat-players could settle out bad cases “where they expected unfavorable rule outcomes” and litigate only the good ones that are “most likely to produce favorable results.” Over time, he concluded, “we would expect the body of ‘precedent’ cases, that is, cases capable of influencing the outcome of future cases – to be relatively skewed toward those favorable to [repeat players].”Footnote ⁷⁸

The Walmart Suite shows that Galanter’s half-century-old prediction is coming to pass, fueled by AI-based software he couldn’t have imagined.Footnote ⁷⁹ And, we anticipate, this isn’t the end of it. In recurring areas of litigation, we are likely to see increasingly sophisticated outcome prediction tools that will draw ever-tighter uncertainty bands around anticipated outcomes. Like the Walmart Suite, these tools are reliant on privileged access to confidential claim settlement data, which only true repeat players will possess.

The effect of this evolution is profound, for, as outcome prediction tools percolate (at least in the hands of repeat defendants/insurers), only duds will be litigated – and this “litigation of losers” will skew – indeed, is almost certainly already skewing – the development of substantive law. The skew will happen because conventional wisdom, at least, holds that cases settle in the shadow of trial – which means that, to the extent trial outcomes tilt toward defendants, we would expect that settlements, too, will display a pro-defendant slant.Footnote ⁸⁰ Damages will also be affected. To offer but one concrete example, in numerous states, a judge evaluates whether damages are “reasonable” by assessing what past courts have awarded for similar or comparable injuries.Footnote ⁸¹ To the extent the repository of past damages reflects damages plaintiffs have won while litigating weak or enfeebled claims, that repository will, predictably, bend downward, creating a progressively more favorable damages environment for defendants.

To be sure, there are caveats and counter-arguments. Models of litigation bargaining suggest that a defendant with privileged information will sometimes have incentives to share that information with plaintiffs in order to avoid costly and unnecessary litigation and achieve efficient settlements.Footnote ⁸² Additionally, while repeat players have better access to litigation and settlement data, even one-shotters don’t operate entirely in the dark.Footnote ⁸³ But a simple fact remains: Even a slow burn of marginally better information, and marginally greater negotiation leverage, can have large aggregate effects across thousands and even millions of cases.

6.3.1 Plausible but Unlikely Reforms

Rewrite substantive or procedural law. First, we could respond to the skew that legal tech brings by recalibrating substantive law. Some combination of state and federal courts and legislatures could, for example, relax liability standards (for instance, making disparate impact job discrimination easier to prove), loosen restrictions on punitive damages, repeal damage caps, return to a world of joint and several liability, restore aider-and-abettor liability, abolish qualified immunity, and resurrect the collateral source rule.

Whatever the merit of a substantive law renaissance, we are, however, quite bearish on the possibility, as the obstacles blocking such an effort are formidable and, over the near- to medium-term, overwhelming. Substantive laws are sticky and salient, especially in a political system increasingly characterized by polarization and legislative gridlock.Footnote ⁸⁴ Even in less-polarized subfederal jurisdictions, it would be hard to convince state legislators and (often elected) judges to enact sweeping reforms without strong support from a public that often clings to enduring but often misguided beliefs about “jackpot justice” and frivolous claiming.Footnote ⁸⁵

Nor are federal courts, or a Supreme Court, newly stocked with Trump-era appointees, likely to help; to the contrary, they are likely to place barriers in front of litigation-friendly legislative efforts.Footnote ⁸⁶ And procedural rules, though less politically salient, will also be hard to change, particularly at the federal level given the stranglehold of conservative judges and defense-side lawyers on the process of court-supervised rulemaking.Footnote ⁸⁷

Democratize the data. Second, we could try to recalibrate the playing field by expanding litigants’ access to currently confidential data. As it stands, when it comes to data regarding the civil justice system, judges, lawyers, litigants, and academics operate almost entirely in the dark. We do not know how many civil trials are conducted each year. We don’t know how many cases go to trial in each case category. And, we don’t know – even vaguely – the outcome of the trials that do take place.Footnote ⁸⁸ Furthermore, even if we could know what happens at trial (which we don’t) or what happens after trial (which we don’t), that still wouldn’t tell us much about the much larger pool of claims that never make it to trial and instead are resolved consensually, often before official filing, by civil settlements.

This is crucial, for without information about those millions of below-the-radar settlements, the ability to “price” a claim – at least to “price” a claim using publicly available data, approaches zero. As Stephen Yeazell has aptly put it:

Recognizing this gap, Computer Sciences Corp. (the maker of Colossus) and Walmart and its tech and BigLaw collaborators are working to fill it. But, they have filled it for themselves – and, in fact, they have leveraged what amounts to their near-total monopoly on settlement data to do so. Indeed, some insurers’ apparent ability to “tune” Colossus rests entirely on the fact that plaintiffs cannot reliably check insurance companies’ work – and so insurers can, at least theoretically, “massage” the data with near impunity.

Seeing the status quo in this light, of course, suggests a solution: We could try to democratize the data. Taking this tack, Yeazell has advocated for the creation of electronic databases whereby basic information about settlements – including, for instance, the amount of damages claimed, the place suit was filed, and the ultimate settlement amount – would be compiled and made accessible online.Footnote ⁹⁰ In the same vein, one of us has suggested that plaintiffs’ attorneys who work on a contingency fee basis and seek damages in cases for personal injury or wrongful death should be subject to significant public disclosure requirements.Footnote ⁹¹

Yet, as much as democratizing the data sounds promising, numerous impediments remain – some already introduced above. The first is that many “cases” are never actually cases at all. In the personal injury realm, for example, the majority of claims – in fact, approximately half of claims that involve represented claimants – are resolved before a lawsuit is ever filed.Footnote ⁹² Getting reliable data about these settlements is exceptionally difficult. Next, even when cases are filed, some high proportion of civil cases exit dockets via an uninformative voluntary dismissal under Rule 41 or its state-level equivalents.Footnote ⁹³ Those filings, of course, may be “public,” but they reveal nothing about the settlement’s monetary terms.Footnote ⁹⁴ Then, even on those relatively rare occasions when a document describing the parties’ terms of settlement is filed with the court, public access remains limited. Despite a brewing “open court data” movement, court records from the federal level on down sit behind “walls of cash and kludge.”Footnote ⁹⁵ Breaking through – and getting meaningful access even to what is “public” – is easier said than done.

“Public option” legal tech. A third unlikely possibility is “public option” legal tech. Perhaps, that is, the government could fund the development of legal tech tools and make them widely available.

When it comes to TAR, public option legal tech is not hard to imagine. Indeed, state and federal judiciaries already feature magistrate judges who, on a day-to-day basis, mainly referee discovery disputes. It may only be a small step to create courthouse e-discovery arms, featuring tech-forward magistrate judges who work with staff technologists to perform discovery on behalf of the parties.

Public option outcome-prediction tools that can compete with Colossus or the Walmart Suite are harder to imagine. Judges, cautious Burkeans even compared to the ranks of lawyers from which they are drawn, are unlikely to relax norms of decisional independence or risk any whiff of prejudgment anytime soon. The bigger problem, however, will be structural, not just legal-cultural. The rub is that, like one-shot litigants and academics, courts lack access to outcome data that litigation’s repeat players possess. Short of a sea change in the treatment of both pre- and post-suit secret settlements, courts, no less than litigation’s have-nots, will lack the information needed to power potent legal tech tools.Footnote ⁹⁶

6.3.2 Slouching Toward Equity: Judicial Procedural Management with an Eye to Technological Realities

Given the above obstacles, the more likely (though perhaps least attractive) outcome is that judges, applying existing procedural rules, will be the ones to manage legal tech’s incorporation into the civil justice system. And, as is often the case, judges will be asked to manage this tectonic transition with few rules and limited guidance, making it up mostly as they go.

The discussion of TAR’s contingent future, set forth above in Part 6.2.2 offers a vivid depiction of how courts, as legal tech’s frontline regulators, might do so adeptly; they might consider on-the-ground realities when addressing and applying existing procedural doctrines. But the TAR example also captures a wider truth about the challenges judges will face. As already noted, legal tech tools that cut litigation costs and hone information derive their value from their exclusivity – the fact that they are possessed by only one side. It follows that the procedural means available to judges to blunt legal tech’s distributive impacts will also reduce the tools’ value and, at the same time, dull incentives for litigants to adopt them, or tech companies to develop them, in the first instance. As a result, disparate judges applying proportionality and work product rules in individual cases will, inevitably, in the aggregate, create innovation policy. And, for better or worse, they will make this policy without the synoptic view that is typically thought essential to making wise, wide-angle judgments.

Judicial management of legal tech’s incorporation into the civil justice system will require a deft hand and a thorough understanding of changing on-the-ground realities. To offer just one example: As noted above, discovery cost concerns have fueled the creation of a number of doctrines that constrict discovery and, in so doing, tend to make life harder for plaintiffs. These include not just Rule 26’s “proportionality” requirement (described above), but also a slew of other tweaks and outright inventions, noted previously, from tightened pleading standards to court-created Lone Pine orders that compel plaintiffs to offer extensive proof of their claims, sometimes soon after filing.

Undergirding all these restrictive doctrines is a bedrock belief: that discovery is burdensome and too easily abused, so much so that it ought to be rationed and rationalized. Yet, as explained above, TAR has the potential to significantly reduce the burden of discovery (particularly, as noted above, if more expansive discovery, which might offset certain efficiency gains, is not forthcoming). As such, TAR, at least arguably, will steadily erode the very foundation on which Twombly, Iqbal, and Lone Pine orders rest – and this newly unsettled foundation might, therefore, demand the reexamination of those doctrines. As judges manage the incorporation of potent new legal tech tools into the civil justice system, we can only hope that they will exhibit the wisdom to reconsider, where relevant, this wider landscape.