Litigation Budget

Subscribe to Litigation Budget

Traditional document review can be one of the most variable and expensive aspects of the discovery process.  The good news is that there are innumerable analytic tools available to empower attorneys to work smarter, whereby reducing discovery costs and allowing attorneys to focus sooner on the data most relevant to the litigation.   And, while various vendors have “proprietary” tools with catchy names, the tools available all seek to achieve the same results:  smarter, more cost effective review in a way that is defensible and strategic.

Today’s blog post discusses one of those various tools – predictive coding.  The next few blog posts will focus on other tools such as email threading, clustering, conceptual analytics, and keyword expansion.

Predictive Coding

Predictive coding is a machine learning process that uses software to take keyword searches / logic, entered by people, for the purpose of finding responsive documents, and applies it to much larger datasets to reduce the number of irrelevant and non-responsive documents that need to be reviewed manually.  While each predictive algorithm will vary in its actual methodology, the process at a very simplistic level involves the following steps:

  1.  Data most likely relevant to the litigation is collected. Traditional filtering and de-duplication is applied.  Then, human reviewers will identify a representative cross-section of documents, known as a “seed set,” from the remaining (de-duplicated) population of documents that need to be reviewed.   The number of documents in that seed set will vary, but it should be sufficiently representative of the overall document population.
  2.  Attorneys most familiar with the substantive aspects of the litigation code each document in the seed set responsive or non-responsive as appropriate. Mind you, many of the predictive coding software available allows users to perform classification for multiple issues simultaneously (i.e., responsiveness and confidentiality).  These coding results will then be input into the predictive coding software.
  3.  The predictive coding software analyzes the seed set and creates an internal algorithm to predict the responsiveness of other documents in the broader population.  It is critically important after this step that the review team who coded the seed set spend time sampling the results of the algorithm on additional documents and refine the algorithm by continually coding and inputting sample documents until desired results are achieved.  This “active learning” is important to achieve optimal results.  Simply stated, active learning is an iterative process whereby the seed set is repeatedly augmented by additional documents chosen by the algorithm and manually coded by a human reviewer. (This differs from “passive learning,” which is an iterative process that uses totally random document samples to train the machine until optimal results are achieved).

Once the team is comfortable with the results being returned, the software applies the refined algorithm to the entire review set and codes all remaining documents as responsive or unresponsive.

 

 

 

 

 

Themis Bar Review, LLC v. Kaplan, Inc., WL 3397877 (S.D. Cal. May 26, 2015).

In this action, defendant served plaintiff with a request for production that sought, among other things, documents related to the plaintiff’s pass rate and the data substantiating the pass rates posted in the plaintiff’s advertising materials. The plaintiff produced the relevant data in a PDF format from Microsoft Excel spreadsheets.  This format, in turn, stripped the documents of all their “user friendly” capabilities (i.e., the ability to filter, sort, and/or search the excel charts).

The defendant complained that the PDF format was not the native format the data was kept in by plaintiff.  Defendant further stated that the plaintiff also failed to respond to any of its many requests during a meet and confer conference wherein defendant asked for the metadata and related email documents.  In response, plaintiff argued that they would comply with defendant’s request to produce the emails and metadata in native form only if defendant paid for the associated costs.

Under Federal Rule Civil Procedure 34, the production of documents or ESI must be produced in “a form or forms in which it is ordinarily maintained or in a reasonably usable form or forms.” As such, the court found that the defendant had notified the plaintiff of the need to produce the spreadsheets and emails in an Excel format, and had tried to meet and confer as contemplated by the parties jointly submitted “Discovery Plan.”

Not surprisingly, given the judicial desire for a cooperative discovery process, the court ordered the plaintiff to reproduce the emails and the associated metadata in searchable format and further ordered plaintiff to pay for the costs of reproduction.  This case, like many others in recent months, serves as a good reminder that often the client is best served by a counsel willing to cooperate during discovery.  What’s more – a cooperative counsel often retains credibility in the eyes of the Court.

In today’s litigious world, discovery is costly and can be perilous. Exacerbating this landscape is the fact that sanctions are imposed for discovery violations more than any other litigation error. Not surprisingly, avoidable discovery mistakes lead to client dissatisfaction.  Below are ten critical tips to avoid discovery sanctions and to remain compliant with discovery obligations.

  1. Implement Timely Litigation Holds Be sure your legal hold is implemented as soon as litigation is reasonably anticipated. Be certain that your hold notice is sufficiently broad, is sent to the right custodians, receipt is acknowledged, and it is updated as needed.
  2. Conduct Key Custodian Interviews A lawyer cannot rely only on the hold notice.  Rather, custodial interviews with key players, IT personnel and anyone else with information relevant to the dispute or the client’s network architecture should be conducted.  Minimally, these interviews will confirm the suspension of auto-delete protocols and will help identify all relevant information for preservation and collection.
  3. Be Proactive Because in today’s technology-intensive world there are substantial quantities of ESI, if you want to receive a document demand before preserving and collecting documents, you may not have time to respond to those demands.  Anticipate document demands so you can start the interview, identification and collection process.  You will have a better handle on the documents (what does and does not exist), and your client’s story such that you will be in the best position to comply with discovery and meet discovery challenges.
  4. Honesty is the Best Policy When Dealing with the Courts and Opposing Parties Never make a factual representation about the status of preservation, collection, or production efforts without confirming the underlying facts with original sources. While a client will rarely mislead their lawyer intentionally, it is common for clients to have incomplete information or operate under a misunderstanding of fact when information is communicated second- hand.   Moreover, courts and opposing parties understand that mistakes can happen at various stages of the discovery process.  Such issues must be addressed immediately and head-on.  Usually the optimal strategy is full disclosure along with remedial measures.
  5. Always Budget Obtain a realistic budget before proceeding with ESI collection processing and/or review.  This is a costly area of litigation and lawyers must manage client expectations. Update the budget as needed to accommodate changes attributable to collection volume or other factors.
  6. You Get More Bees with Honey… Seek a cooperative approach irrespective of how unpleasant or unreasonable opposing counsel may be. Indeed, a cooperative approach to discovery will invariably reduce disputes and expenses. Take the higher road and assume that every email and letter you write to opposing counsel may end up in front of the judge, so adopt a cooperative approach and reasonable tone in all communications with opposing counsel.    As one of our earlier blog posts showed (see Armstrong Pump, Inc. v. Hartman, No. 10-CV-446S, 2014 WL 6908867 (W.D.N.Y. Dec. 9, 2014)), Judges have very little patience for uncooperative behavior during a lawsuit’s “search for the truth.”
  7. There’s No Longer Room For Boilerplate Discovery The amended FRCP 26(g)(1)(B)(iii) provides that every discovery request and response must be signed by at least one attorney of record, and by signing you certify that the discovery request or response is proportional – meaning “neither unreasonable nor unduly burdensome or expensive considering the needs of the case, prior discovery in the case, the amount in controversy, and the importance of issues at stake….”  The Rule goes on to state that “[i]f a certification violates this rule without substantial justification, the court must impose an appropriate sanction on the signer, the party on whose behalf the signer was acting, or both.”
  8. Be Careful What You Wish For…Lest You Receive It In Return Never send a discovery request to an adversary that you or your client would be uncomfortable complying with were opposing counsel to author a reciprocal request to you.
  9. Carefully Devised Search Terms Are Critically Important The judgment of your legal team is a good starting point for crafting search terms, but is far from sufficient.  Review a preliminary “hit-by-term” report from your ESI vendor so you can appreciate which terms are too limiting or overbroad.  During custodial interviews (see supra) ask about project code names, and other unique search terms.  Then sample, sample, sample!  Sampling the documents—both the hits and the non-hits—can help refine search terms and validate the terms chosen.
  10. Wise Use of Technology Can Be a Litigator’s Best Friend ESI processing, review (even with contract attorneys) and production is among the most costly elements of any litigation.  When used efficiently and wisely, technology can significantly reduce those costs. Consider early data assessment, filtering and predictive coding technology as appropriate for each matter.