Discovery

Subscribe to Discovery

Traditional document review can be one of the most variable and expensive aspects of the discovery process.  The good news is that there are innumerable analytic tools available to empower attorneys to work smarter, whereby reducing discovery costs and allowing attorneys to focus sooner on the data most relevant to the litigation.   And, while various vendors have “proprietary” tools with catchy names, the tools available all seek to achieve the same results:  smarter, more cost effective review in a way that is defensible and strategic.

Today’s blog post discusses one of those various tools – predictive coding.  The next few blog posts will focus on other tools such as email threading, clustering, conceptual analytics, and keyword expansion.

Predictive Coding

Predictive coding is a machine learning process that uses software to take keyword searches / logic, entered by people, for the purpose of finding responsive documents, and applies it to much larger datasets to reduce the number of irrelevant and non-responsive documents that need to be reviewed manually.  While each predictive algorithm will vary in its actual methodology, the process at a very simplistic level involves the following steps:

  1.  Data most likely relevant to the litigation is collected. Traditional filtering and de-duplication is applied.  Then, human reviewers will identify a representative cross-section of documents, known as a “seed set,” from the remaining (de-duplicated) population of documents that need to be reviewed.   The number of documents in that seed set will vary, but it should be sufficiently representative of the overall document population.
  2.  Attorneys most familiar with the substantive aspects of the litigation code each document in the seed set responsive or non-responsive as appropriate. Mind you, many of the predictive coding software available allows users to perform classification for multiple issues simultaneously (i.e., responsiveness and confidentiality).  These coding results will then be input into the predictive coding software.
  3.  The predictive coding software analyzes the seed set and creates an internal algorithm to predict the responsiveness of other documents in the broader population.  It is critically important after this step that the review team who coded the seed set spend time sampling the results of the algorithm on additional documents and refine the algorithm by continually coding and inputting sample documents until desired results are achieved.  This “active learning” is important to achieve optimal results.  Simply stated, active learning is an iterative process whereby the seed set is repeatedly augmented by additional documents chosen by the algorithm and manually coded by a human reviewer. (This differs from “passive learning,” which is an iterative process that uses totally random document samples to train the machine until optimal results are achieved).

Once the team is comfortable with the results being returned, the software applies the refined algorithm to the entire review set and codes all remaining documents as responsive or unresponsive.

 

 

 

 

In this single-plaintiff employment discrimination case (Bailey v. Brookdale Univ. Hosp., 2017 U.S. Dist. LEXIS 93093 (E.D.N.Y. June 16, 2017)), counsel for the parties purportedly met and conferred as directed by the Court and, thereafter, entered into an ESI agreement (“Agreement”).  The Agreement was presented to the Court and represented to be the product of mutual negotiation.  As a result, the Court So-Ordered the Agreement and its terms.

During discovery, Bailey advised the Court that he was no longer able to comply with the Agreement because the data production costs would cause an economic hardship. Specifically, he claimed the cost of production – estimated at $2,000-$3,000 – was unduly burdensome in light of his personal financial situation, notwithstanding the Agreement.  At the Court’s request, Bailey submitted an affidavit estimating the cost of production and that such a cost would inflict a “severe financial hardship” on him given that he earned approximately $90,000 annually and was the sole provider for his family of five.  In evaluating Bailey’s grievance, the Eastern District considered cost-shifting to protect Bailey from incurring an undue burden or expense. For cost-shifting to be properly granted, however, there must be sufficient proof of economic hardship and evidence that the requested data is inaccessible.  The Court found neither was established by Bailey. Nonetheless, the Court found that the Agreement proposed by Defendants was of a type, “typically utilized in a more complex litigation involving multiple parties and corporate entities” and had no applicability to a single plaintiff.   As a result, the Court concluded that Bailey’s counsel did not engage in meaningful discussions with his client regarding the terms of the proposed Agreement and what costs might be incurred by producing the information in the format the defendants sought. Likewise, it further appeared to the Court that Bailey’s counsel did not engage in a meaningful meet-and-confer session with opposing counsel, and did not thoroughly review the Agreement prior to signing it.

The Court did not find sufficient grounds to terminate the Agreement, and instead ordered partial cost-shifting [so that Defendants received the form of production they negotiated for], requiring the defendants to bear 40% of discovery costs and Bailey’s counsel, rather than Bailey himself, to bear the remaining 60%.

This case serves as an important reminder of counsels’ obligation to engage in good faith in all aspects of the discovery process – including negotiating an ESI production protocol.  Here, the Court was unwilling to revise the Agreement, and instead required Bailey’s counsel to abide by the terms of the Agreement and pay for the production.  This case also serves as an important reminder of our duty, as lawyers, to be competent in the law and the technological world in which we practice.  Indeed, as attorneys practicing in today’s ever-increasingly electronic world, we must remain abreast of the intricacies involved in electronic production and the costs associated with that ESI.  (See earlier blogs discussing duty of competence)

Armstrong Pump, Inc. v. Hartman, No. 10-CV-446S, 2016 WL 7208753 (W.D.N.Y. Dec. 13, 2016)

In this case, pending before the Court was a motion by Armstrong Pump Inc. (“Armstrong”) to compel formal production of certain documents that defendant Optimum Energy LLC (“Optimum”) considered the functional equivalent of its proprietary source code.  This “formal production” Armstrong sought to compel consisted of 46 documents (305 pages) that Armstrong previously viewed during a “document and source code review” conducted pursuant to a strict protocol.  During that review Armstrong was permitted to attend with two outside counsel and two experts.  Moreover, Armstrong was able to print – also under strict protocol – certain of those documents.

In support of its motion to compel, Armstrong now argues that the reviewed documents did not contain “actual programming” and thus were able to be produced under the parties’ protective order for discovery without any additional protections designed to protect source code. Optimum argued in response that the documents were “functionally equivalent to source code” and should not be subject to production.  Specifically, Optimum contended that the documents contained enough technical information including (in detail) the functions, logic and algorithms implemented in Optimum’s products to “allow a software engineer to build its proprietary source code based on that information.”

Ultimately, the Court reasoned that discovery had “reached the point of diminishing returns” and declined to compel production, with limited exceptions.

In reaching this conclusion, the court first addressed the effects of the 2015 amendments to Federal Rule 26, reasoning that “proportionality has assumed greater importance in discovery disputes” and that the amended rule is intended to encourage more aggressive efforts from the judiciary to discourage discovery overuse. The court continued:

Discouraging discovery overuse does not end with the early stages of a case, however. Implicit in both the language and the spirit of the 2015 Amendments is the obligation, at any stage of a case, to prevent parties from expending increasing time and energy pursuing diminishing returns. Calling a halt to the pursuit of diminishing returns often will overlap with an assessment of duplicate or cumulative discovery. Sometimes the additional discovery sought technically would provide nominally probative information not yet in the parties’ hands. Either way, when adding a few more pages of documents requires five or six inches of motion papers, and when those few more pages would be added to over one million pages of total discovery, numerous pages of expert reports, and transcripts from depositions of all of the relevant players, there exists a point beyond which courts have to tell the parties that if they cannot yet prove their claims then they probably never will. See Alaska Elec. Pension Fund v. Bank of Am. Corp., No. 14-CV-7126 (JMF), 2016 WL 6779901, at *3 (S.D.N.Y. Nov. 16, 2016) (“Rule 26(b)(1)’s proportionality requirement means [that a document’s] ‘marginal utility’ must also be considered.”) (citations omitted); Updike v. Clackamas County, No. 3:15-CV-00723-SI, 2016 WL 111424, at *1 (D. Or. Jan. 11, 2016) (“There is a tension, however, among the objectives of Rule 1. As more discovery is obtained, more is learned. But at some point, discovery yields only diminishing returns and increasing expenses. In addition, as more discovery is taken, the greater the delay in resolving the dispute. Finding a just and appropriate balance is the goal, and it is one of the key responsibilities of the court in managing a case before trial to assist the parties in achieving that balance.”).

The court then turned to the discovery specifics in the underlying case and concluded that discovery had “reached the point of diminishing returns.” Indeed, discovery had been ongoing for six years and the parties exchanged more than 1,600,000 pages of documents.  The Court listed a litany of critical case development failures along the 6 year discovery path and concluded it hard to believe that 150 pages of already reviewed but not printed documents would definitively prove what six years of discovery could not.

This case serves as a further reminder that discovery is not limitless.  Indeed, there is little tolerance for cumulative discovery and proportionality is the key inquiry under new federal landscape.